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Abstract 

I^The  objective  of  this  study  was  to  develop  an 
acquisition  management  tool  that  would  improve  the  working 
relationship  between  the  System  Program  Office  (SPO)  and  the 
User  during  the  Demonstration  and  Validation  phase.  It 
accomplished  this  by  documenting  the  Advanced  Tactical 
Fighter  (ATF)  Early,  Structured,  and  Continuous  (ESC)  user- 
interface  process  in  a  draft  handbook.  This  handbook  was 
also  developed  to  support  a  continuing  Command-wide  effort 
to  improve  the  acquisition  process. 

The  study  found  that  the  ATF  ESC  process  was  also 
developed  in  response  to  senior  Air  Force  management 
guidance,  lessons  learned,  and  ATF  SPO  management  direction. 
The  ATF  ESC  process  was  easily  incorporated  into  the  SPO's 
acquisition  strategy,  and  it  did  not  affect  the  current 
Tactical  Air  Force  requirements  process  and  the  player's 
roles  and  responsibilities.  It  has  also  been  in  operation 
for  four  years,  but  was  not  documented. 

The  study  recommended  that  the  handbook  be  formalized 
and  provided  to  other  SPOs  as  a  management  tool  for 
establishing  an  effective  SPO-User  relationship/^  The 
handbook  could  be  modified  and  used  as  a  SPO  training  tool 
with  additional  research.  The  handbook  could  also  be 
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modified  to  address  the  AFSC/AFLC  merger  as  well  as  FSED, 
but  additional  research  would  need  to  be  accomplished. 
Finally,  after  the  ATF  program  is  two  to  three  years  into 
Full  Scale  Development,  the  effectiveness  of  the  ATF  ESC 
process  could  be  measured  against  the  F-15  and  F-16  programs 
by  comparing  the  number  and  type  of  changes  to  the 
technical /operational  characteristics  documented  in  the 
Selected  Acquisition  Report  (SAR). 
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THE  ADVANCED  TACTICAL  FIGHTER  EARLY,  STRUCTURED,  AND 


CONTINUOUS  USER- INTERFACE  PROCESS:  A  SYSTEM 
PROGRAM  OFFICE/USER  TEAM-BUILDING  CONCEPT 


I .  Introduction 


Background 

The  declining  defense  budget  and  the  rising  cost  of 
weapon  systems  have  forced  Air  Force  decision  makers  to 
reduce  the  force  structure  and  emphasize  technological 
solutions.  This  has  complicated  the  process  of  translating 
operational  requirements  into  a  weapon  system  design  through 
the  requirements  generation  process.  In  addition,  changing 
threats,  program  budgets,  and  program  schedules  continue  to 
press  the  System  Program  Office  (SPO)  and  the  user  to  refine 
or  even  modify  the  initial  requirements,  especially  in  the 
early  phases  of  an  acquisition  program.  Therefore,  it  is 
important  that  the  SPO  and  the  user  develop  and  maintain  an 
early,  structured,  and  continuous  interface  so  that  there  is 
a  match  between  the  requirements  and  achievable  technologies 
to  baseline  an  executable  program  by  Full  Scale  Engineering 
Development  (FSED)  (1). 

General  Issue 

The  Acquisition  Process  Excellence  (APEX)  committee  was 
established  in  October  1989  by  General  Bernard  P  Randolph, 
former  commander  of  Air  Force  Systems  Command  (AFSC). 


APEX  was  a  mul ti-f iinctional  team  consisting  of 
representatives  from  each  AFSC  field  organization,  HQ  AFSC, 
SAF/AQ,  and  the  user.  The  mission  of  APEX  was  to  create  a 
revolutionary  team  acquisition  process  that  destroyed 
barriers,  inspired  opportunity  for  improvement,  and 
increased  responsiveness  to  the  user  by  baselining, 
evaluating,  and  identifying  the  "best  practices"  in  any 
acquisition  program  throughout  the  Command  (15). 

The  first  phase  of  the  study  was  completed  in  March 
1990.  The  committee  concluded  that  the  process  of 
developing  acquisition  strategies  lacked  a  consistent, 
systematic  approach,  was  not  responsive  to  the  user,  did  not 
comprehensively  address  program  risks,  and  resulted  in  a 
lack  of  corporate  commitment  and  inexecutable  programs  (15). 
In  addition,  during  the  APEX  teams's  data  collection  phase, 
which  involved  surveying  program  managers  and  functional 
experts  across  the  Command,  recurring  inputs  on  perceived 
acquisition  problems  centered  on  ways  to  improve  the 
relationship  between  AFSC  and  the  user  (15). 

The  APEX  team  recommended  a  shift  in  planning  emphasis 
to  the  front  end  of  all  system  acquisitions,  and  the 
development  of  more  effective  team  processes  that  would 
optimize  the  contributions  and  roles  of  all  the 
participants-users ,  contractors,  program  managers,  and 
functional  managers-in  the  acquisition  process  (15).  The 
Advanced  Tactical  Fighter  (ATF)  SPO  has  developed  a  similar 
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process,  the  Early,  Structured,  and  Continuous  (ESC)  user- 
interface  process,  for  improving  their  relationship  with  the 
user  as  well  as  other  participants.  This  has  resulted  in  a 
highly  integrated,  coordinated,  and  effective  team 
acquisition  process  for  the  Demonstration  and  Validation 
Phase  of  their  program.  It  also  resulted  in  a  stable, 
achievable  design,  which  meets  the  user's  requirements,  at 
the  end  of  this  phase.  It's  been  in  operation  for  four 
years  and  it  was  supported  by  Senior  Air  Force  management, 
but  it  has  not  been  documented. 

ATF  Program  Overview 

The  ATP  is  being  developed  as  an  Air  Superiority 
fighter  to  dominate  the  threat  in  the  year  2000  and  beyond 
with  a  primary  design  mission  of  offensive  counterair  (13). 
Developing  a  system  with  improved  performance 
characteristics,  such  as  supersonic  persistency,  high 
maneuverability,  and  better  combat  radius,  within 
congressional ly  mandated  goals  for  a  50,000  pound  aircraft 
at  35  million  dollar  average  unit  flyaway  costs  (base  year 
1985  dollars),  was  a  challenging  task  requiring  optimization 
of  numerous  variables  which  must  be  treated  in  an  integrated 
analysis  (13). 

The  program  just  completed  a  competitive  Demonstration 
and  Validation  (D/V)  phase.  During  the  D/V  phase,  the  most 
promising  technical  alternatives  from  the  Conceptual 
Exploration/  Demonstration  (CE/D)  are  refined  by  making 
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performance  tradeoffs  whereby  high-cost  features  providing 
only  marginal  performance  gains  are  deleted  from  the  system 
(37:301).  It  will  enter  Full  Scale  Development  (FSD)  in 
mid-1990.  Production  options  are  scheduled  to  be  exercised 
beginning  in  1996  with  the  Tactical  Air  Force  receiving 
aircraft  beginning  in  1999. 

ATF  ESC  User-Interface  Process 

The  ATF  SPO  developed  the  ESC  user-interface  process  in 
response  to  a  need  for  a  better  relationship  between  the  SPO 
and  the  user  in  their  D/V  phase.  The  relationship  was 
critical  during  this  phase  because  the  SPO  and  the  user  had 
the  greatest  opportunity  to  change  the  weapon  system  design 
before  it  was  baselined  in  Full  Seals  Engineering 
Development  (FSED)  (23).  The  primary  objectives  of  the  ATF 
ESC  process  were  to  add  justification  and  accountability  to 
the  requirements  process  by  involving  and  educating  the  User 
during  the  trade  studies  process,  and  to  ensure  their 
continued  support  in  a  very  political  program  (1). 

The  user  was  involved  in  every  aspect  of  the  program  to 
ensure  the  operational  requirements  became  an  integral  part 
of  the  weapon  system  design  process.  The  degree  of 
interaction  between  the  User  and  the  engineering  communities 
was  unparalleled.  Representatives  from  the  TAF,  Air  Force 
Logistics  Command  (AFLC),  and  Air  Training  Command  (ATC) , 
for  the  ATF,  were  collocated  in  the  program  office  and 
treated  as  members  of  the  SPO.  Formal  agreements,  such  as 
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Memo  rand  vans  of  Agreements  (MOAs),  were  not  needed  due  to  the 
process's  support  from  senior  Air  Force  management.  The  ATF 
SPO  established  a  team  of  experts  for  Operations  and  Support 
(O&S)  and  Manpower,  Personnel,  Training,  and  Safety  (MPTS) 
functions  within  the  SPO  to  ensure  that  they  were  integrated 
into  the  system  design.  The  User  attended  working  groups, 
Technical  Interchange  Meetings  (TIMs),  program  reviews,  and 
briefings  at  all  levels.  The  user  participated  in  Mission 
Area  Analysis/Cost  and  Operational  Effectiveness  Analysis 
(MAA/COEAs)  and  other  ATF-related  studies. 

Specific  Problem 

Historically,  the  user  has  inadvertently  been  excluded 
from  the  acquisition  process  after  the  approval  of  the 
Mission  Element  Need  Statement  (MENS),  or  became  proactive 
after  the  weapon  system  design  was  frozen.  Typically, 
therefore,  the  process  of  identifying  and  refining  weapon 
system  requirements  through  the  trade  study  process  is  not 
responsive  to  program  changes,  and  the  results  lack 
justification  and  traceability  because  SPOs  have  not 
developed  and  maintained  an  early,  structured,  and 
continuous  interface  with  the  user. 

The  APEX  team  identified  and  validated  a  number  of  key 
areas  related  to  the  requirements  generation  process,  where 
the  SPO  and  user  relationship  could  be  improved  to  make  the 
acquisition  process  more  effective.  They  were:  cooperation 
and  exchange  of  ideas,  disciplined  requirements  definition. 
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team  approach.  Mission  Area  Analysis  and  Operational  Cost 
and  Effectiveness  Analysis  (MAA/COEA) ,  concept  DIR  studies, 
standard/  simplified  approach  to  major  and  non-major 
programs,  exclusion  from  acquisition  process,  and  improved 
synergism  of  documentation  (16).  The  Team's  recommendation, 
the  Early  Acquisition  Strategy,  addressed  these  concerns, 
but  it  has  not  been  implemented. 

The  APEX  committee  continues  to  work  with  functional 
experts  and  program  managers  throughout  the  Command  to 
identify  the  "best  practices"  in  any  acquisition  program  to 
enhance  the  acquisition  process.  Also,  the  EAS  strategy  may 
not  be  implemented  for  some  time  because  it  represents  a 
major  change  to  the  present  Air  Force  weapon  system 
acquisition  process. 

Research  Objective 

The  objective  of  this  study  was  to  develop  an 
acquisition  management  tool  that  would  improve  the 
relationship  between  the  SPO  and  the  User  during  the 
acquisition  process,  so  that  the  trade  study  process  could 
be  improved.  This  was  accomplished  by  documenting  the  ATF 
ESC  user-interface  process  in  a  draft  handbook.  This 
handbook,  the  product  of  a  comprehensive  study  of  the  ATF 
ESC  interface  process  with  the  User,  was  developed  to 
support  a  Command-wide  effort  to  improve  the  acquisition 
process.  The  handbook  identified  specific  management  and 
engineering  techniques  that  were  developed  by  the  ATF  SPO 
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and  the  user,  and  which  fall  under  the  following  concern 
areas  from  the  APEX  study:  cooperation  and  exchange  of 
ideas,  disciplined  requirements  definition,  team  approach, 
exclusion  from  acquisition  process,  and  improved  synergism 
of  documentation. 

The  handbook  will  also  include  possible  improvements  to 
the  ATF  approach,  as  perceived  by  four  selected  groups.  The 
four  groups  included  the  APEX  team,  ATF  SPO,  user,  and 
contractor  personnel . 

The  population  consisted  of  APEX  team  members,  program 
directors,  and  functional  experts  involved  in  the  APEX 
study.  Personnel  from  the  ATP  SPO  included  SPO  management, 
functional  experts,  and  Tactical  Air  Command  Liaison  Officer 
(TACLO).  Personnel  from  the  user  included  TAC/DR,  the  APEX 
team,  and  AFSC.  The  defense  contractors  included  the 
managers  involved  in  the  ATP  user-interface  process,  as  well 
as  prime  contractors'  OPR's. 

Investigative  Questions 

The  task  of  developing  the  handbook  inspired  some 
specific  questions  which  guided  the  research.  These 
questions  are  presented  next: 

1.  What  factors  caused  the  ATF  SPO  to  develop  the  ESC 
interface  process  with  the  user? 

2.  What  changes  were  required  in  the  ATF  SPO's 
acquisition  strategy  to  support  the  ESC  interface  process 
with  the  user? 
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3.  What  engineering  and  management  techniques  were 
developed  by  the  ATF  SPO  for  the  ESC  interface  process  with 
the  user? 

4.  How  can  these  engineering  and  management  techniques 
be  generalized  and  applied  to  other  major  Air  Force  acquisi¬ 
tion  programs? 

5.  Into  which  acquisition  phase  should  the  program 
director  implement  the  ATF  ESC  interface  process  with  the 
user? 

6.  What  approval  cycle  is  required  to  implement  the 
ATF  SPO  ESC  interface  process  with  the  user? 

Scope 

This  study  was  designed  to  support  a  continuing, 
command-wide  effort  to  improve  the  acquisition  process  and 
critical  subprocesses.  The  goal  was  to  document  the  ATF  ESC 
process  and  provide  a  management  tool  to  other  Air  Force 
acquisition  programs  in  this  area. 

This  was  accomplished  through  a  literature  review  of 
all  acquisition  improvement  strategies  and  implementation 
plans,  including  the  ATF  ESC  user-interface  process,  as  well 
as  personnel  interviews.  Initially,  this  study  used  the 
Defense  Technical  Information  Center  (DTIC)  and  professional 
journals  from  the  Air  Force  Institute  of  Technology  (AFIT) 
library.  Later,  Advanced  Tactical  Fighter  program 
documentation.  Air  Force  studies.  Air  Force  policies  and 
documentation,  past  theses,  and  interviews  were  the  main 
sources  of  information  due  to  the  narrow  scope  of  this  study 
and  the  underlying  assumptions. 
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Assumptions 

The  handbook  was  designed  as  a  management  tool  for 
establishing  an  interface  process  with  the  user. 
Consideration  should  be  given  to  the  following  assumptions 
when  planning  and  implementing  a  user-interface  process 
based  on  some  of  the  ATF  concepts: 

1.  Based  on  draft  Defense  Management  Review  (DMR) 
directions,  a  SPO  will  not  be  formed  xintil  after  Milestone  I 
(15). 


2.  User  deficiencies/needs  were  validated  by  the 
Milestone  0  decision  (15). 

3.  Concept  Exploration  and  Exploration  funding  pool 
was  available  to  fund  the  concept  direction  studies  and 
development  of  alternative  technical  solutions  (15). 

4.  The  primary  participants  in  the  ESC  process  were 
the  SPO,  user,  contractor,  HQ  APSC,  and  HQ  USAF. 

5.  The  customers  of  process  were  the  user,  DAB  II 
process,  D/V  program  directors.  Science  Advisory  Boards 
(SAB),  System  Design  Reviews  (SDRs),  Mission  Area 
Analysis/Cost  and  Operational  Effectiveness  Analysis 
(MAA/COEAS),  program  reviews,  TIMs,  and  working  groups. 

6.  The  ATF  ESC  user-interface  process  is  planned  and 
implemented  in  the  D/V  phase. 

7.  Major  programs  were  addressed,  but  the  philosophy 
may  be  applicable  to  less-than  major  programs. 

8.  The  SPO  specified  the  requirements  in  the  form  of  a 
threat  description  and  preliminary  system  specification. 

9.  The  acquisition  strategy  involved  teaming 
arrangements,  competition,  and  prototyping. 


Limitations 

The  ATP  ESC  SPO-User  team-building  handbook  discusses 
specific  management  and  engineering  activities  that  were 
implemented  by  the  ATP  SPO  in  response  to  a  Command-wide 
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need  for  a  better  relationship  between  the  SPO  and  the  User 
during  the  acquisition  process.  The  handbook  is  in  draft 
form.  Currently,  it  is  applicable  to  the  D/V  phase  of  major 
acquisition  programs.  The  acquisition  process  is  presented 
from  an  ATF  SPO  viewpoint  including:  roles  and 
responsibilities  of  the  Government  and  contractors; 
documentation  generated  by  the  SPO;  and  objectives  of  the 
acquisition  phases.  Therefore,  if  the  handbook  is  used  as  a 
management  tool,  one  should  consider  this  as  well  as  the 
underlying  assumptions. 

Justification 

The  ATF-based  handbook  could  be  delivered  directly  to 
the  field  organizations  without  additional  AFSC  action  or 
changes  to  the  current  TAP  requirements  process  philosophy. 

It  focused  on  D/V  activities  critical  to  the 
acquisition  process  rather  than  activities  associated  with 
Mission  Area  Analysis  (MAA)  and  the  CE/D  phase  like  previous 
user-based  strategies.  Therefore,  it  represents  a  different 
look  at  the  acquisition  process. 

For  example,  in  the  APEX  study,  a  new  acquisition 
strategy  process  was  defined  and  later  called  the  Early 
Acquisition  Strategy.  This  Early  Acquisition  Strategy 
process  focused  on  the  activities  that  need  to  be  conducted 
during  the  CE/D  phase  while  this  study  focused  on  the 
activities  in  the  D/V  phase  (15). 
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Early  involvement  of  the  using  and  supporting  command 
personnel  greatly  contributed  to  the  success  of  the  F-IS 
program.  From  early  in  the  CE/D  phase,  TAG,  AFLC,  and  ATC 
were  involved  during  the  requirements  formulation  and 
Request  for  Proposal  (RFP)  preparation  (37:293).  They  were 
collocated  within  the  SPO,  and  became  an  integral  part  of 
the  P-15  team.  Unfortunately,  this  user-interface  process 
which  was  used  in  the  CE/D  phase  was  never  documented. 

Finally,  the  Crew  Centered  Cockpit  Design  (CCCD) 
Advanced  Development  Program  Office  (ADPO),  OL-AC  HAD/YAH 
(CAT),  at  Wright-Patterson  APB,  Ohio,  completed  a  study 
that : 

Developed  a  design  process  that  emphasizes  an  early 
employment  and  more  even  distribution  of  resources  to 
converge  on  a  crew  system  design.  Thus,  this  effort 
described  what  activities  are  required  for  crew  system 
acquisition  (which  is  applicable  to  all  acquisitions), 
but  not  how  the  activities  are  performed  (which  would 
vary  between  the  different  programs.  (14:Vol  1,1) 

The  CAT  model  could  be  used  as  an  acquisition  training  tool 

for  the  SPO  and  the  user,  but  was  not  considered  in  this 

study . 

System  Acquisition  Process 

Although  this  study  concentrated  on  the  D/V  phase  of 
the  acquisition  process,  it  is  necessary  to  understand  the 
objectives  of  the  other  acquisition  phases. 

Briefly,  weapon  system  programs  go  through  a  sequence 
of  key  program  decisions  and  milestones  known  as  the  systems 
acquisition  process  or  the  acquisition  life  cycle  (37:258). 


The  acquisition  process  is  a  logical  flow  progressing  from 
the  identification  of  a  system  need  to  operational 
deployment  and  support  of  the  new  system  (37:258). 

Operational  Requirements.  The  operational  requirements 
process  is  a  structural  process  of  analysis  and  iterative 
refinement  by  which  HQ  USAF  and  the  User  can  document 
operational  deficiencies  and  needs  for  validation  and 
solution  (37:301).  When  the  analysis,  otherwise  known  as 
mission  area  analysis,  identifies  a  deficiency  in  existing 
agency  capabilities  or  an  opportunity  to  establish  new 
capabilities  in  response  to  a  technologically  feasible 
opportunity,  this  will  be  formally  set  forth  in  a  mission 
element  need  statement  (MENS)  (37:302). 

Approval  of  the  MNS  starts  the  major  system  acquisition 
process  by  granting  authority  to  explore  alternative  system 
design  concepts.  This  initial  approval  and  the 
establishment  of  a  system  acquisition  program  does  not 
automatically  mean  that  a  new  major  system  will  eventually 
be  acquired.  With  an  approved  need,  designated  agency 
component(s)  may  continue  to  analyze  other  optional  means  of 
satisfying  the  need  in  parallel  with  the  exploration  of 
alternative  systems  which  may,  as  development  proceeds, 
prove  unacceptable. 

Concept  Exploration  and  Definition  Phase.  Once  the 
need  is  established,  a  new  weapon  system  enters  the  CE/D 
phase.  The  objective  of  this  phase  is  to  explore  material 
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alternatives  to  satisfying  the  documented  mission  need. 

This  phase  generally  consists  of  paper  studies,  but  limited 
experiments  and  tests  may  be  performed. 

Selection  of  the  system  concepts  that  offer  the  best 
balance  of  cost,  schedule,  and  technical  performance  will  be 
made  during  this  stage.  If  the  selected  alternatives 
warrant  system  demonstration,  then  approval  of  the  SECDEF 
must  be  obtained  before  proceeding  to  the  next  phase. 

Demonstration  and  Validation  Phase.  In  the  D/V  phase, 
selected  alternatives  from  the  previous  phase  are  further 
evaluated  and  defined.  The  central  thrust  of  the  effort 
during  this  phase  is  reduction  of  risk  and  economic 
uncertainty  and  a  more  detailed  definition  of  the  new  system 
(37:262).  It  is  important  to  get  a  handle  on  the 
requirements  because  the  airframe  cannot  be  changed  much  in 
PSED  (23).  It  should  be  emphasized  that  any  prototyping  at 
this  point  only  resembles  the  operational  system  to  the 
point  that  performance  testing  and  evaluation  can  be 
performed . 

Once  this  phase  is  completed,  the  results  are  forwarded 
through  the  appropriate  channels  to  the  SECDEF  for  approval 
to  proceed  to  the  next  phase. 

Full  Scale  Engineering  and  Development  Phase.  During 
this  phase  of  the  acquisition  life  cycle,  the  system  is 
designed,  developed,  fabricated,  and  tested.  At  the  end  of 
this  phase,  the  design  specifications  and  the  engineering 
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drawings  will  be  finalized.  Testing  and  Evaluation  (T&E) 
are  an  important  part  of  this  phase.  Through  rigorous  T&E, 
the  contractor  and  the  Air  Force  identify  and  solve 
engineering  problems  (37:263).  Furthermore,  the  T&E 
demonstrate  that  program  objectives  have  been  met  and  that 
continuation  to  production  is  warranted. 

Production  and  Deployment  Phase.  During  this  phase, 
the  system,  including  training  equipment,  spares, 
facilities,  etc  is  produced  for  operational  use.  Testing  is 
continued,  and  the  system  is  integrated  into  an  as  close  as 
an  operational  configuration  as  possible. 

Deployment  begins  when  the  weapons  are  provided  to  the 
operating  command.  The  operating  command  then  has  the 
responsibility  to  assume  operation  and  maintenance  and 
assume  property  accountability.  This  point  is  identified  as 
initial  operational  capability  (IOC). 

Operation  and  Support  Phase.  During  this  phase,  the 
user  identifies  shortcomings  or  deficiencies  that  must  be 
corrected  to  improve  performance.  In  addition,  the  other 
commands  ensure  the  fielded  system  continues  to  provide  the 
capabilities  to  meet  the  identified  mission  need. 

Definitions 

In  order  to  understand  any  aspect  of  the  acquisition 
process,  which  includes  the  operational  requirements 
process,  it  is  necessary  to  familiarize  the  reader  with  some 
related  terminology. 
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1.  Acquisition  process.  A  capstone  term  used  to  cover 
the  entire  process  of  acquiring  weapons  from  requirement 
generation,  research,  development,  test  and  evaluation, 
production,  operational  support,  product  improvement,  and 
final  disposal  (37:258). 

2.  Air  Force  Logistics  Command  (AFLC) .  Major  Air 
Force  command  responsible  for  logistical ly  supporting  all 
Air  Force  activities  as  well  as  other  DoD  and  federal 
agencies.  The  supporting  command  (19:68).  Also  known  as 
the  Supporting  Command. 

3.  Air  Force  Systems  Command  (AFSC).  Major  Air  Force 
command  responsible  for  total  Air  Force  research, 
development,  testing,  evaluation,  and  contracting  fo’’ 
production  of  aerospace  systems  (14:Vol  VII, 2). 

4.  Air  Training  Command  (ATC).  Major  air  command 
responsible  for  training  aircrews  and  maintenance  personnel. 
The  participating  command. 

5.  Aeronautical  Systems  Division  (ASD) .  Division  of 
Air  Force  Systems  command  responsible  for  directing  the 
design,  development,  and  acquisition  of  major  aerospace 
systems. 


6.  Aeronautical  Systems  Division/YF.  The  Advanced 
Tactical  Fighter  System  Program  Office.  Organization  within 
ASD  responsible  for  managing  the  design,  development,  and 
acquisition  of  the  advanced  tactical  fighter. 

7.  Concept  Exploration/Definition  (CE/D)  phase. 

Weapon  system  acquisition  phase  where  a  SPO  cadre  explores 
generic  design  alternatives  to  satisfy  a  stated  mission 
need. 


8.  Cockpit  Automation  Technology  (CAT).  Advanced 
development  program  responsible  for  development  of  highly 
disciplined  and  structured  crew  system  design  procvjss 
including  the  supporting  design  tools  and  the  technology 
(14:Vol  VII, 8). 

9.  Defense  Acquisition  Board  (DAB).  Top  level  DoD 
corporate  body  for  system  acquisition  responsible  for 
providing  advice  and  assistance  to  the  Secretary  of  Defense. 
Reviews  the  weapon  system  acquisition  at  milestone  decision 
points  (14:Vol  VII, 22). 

10.  Demonstration/Validation  (D/V)  phase.  Weapon 
system  phase  where  program  characteristics  (performance, 
cost,  and  schedule)  are  validated  and  refined  through 
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extensive  study  and  analysis,  hardware  development  and 
prototype  testing  (14:Vol  VII, 22). 

11.  Full  Scale  Engineering  Development  (FSED).  The 
weapon  system  acquisition  phase  where  the  weapon  system  is 
designed,  fabricated,  and  tested. 

12.  Milestone.  Major  program  decision  points 

which  require  Secretary  of  Defense  approval  to  proceed  with 
the  weapon  system  acquisition.  Milestone  0  proceeds  CE/D 
phase.  Milestone  I  proceeds  D/V  phase.  Milestone  II 
precedes  FSED  (14:Vol  VII, 51). 

13.  Mission  Need.  Deficiencies  (threat  changes, 
decreased  performance  of  older  systems,  or  national 
security  policy  changes),  technological  opportunity  or 
expanded  mission  which  created  the  need  for  a  new  or 
modified  weapon  system  (14:Vol  VII, 51). 

14.  Mission  Element  Need  Statement  (MENS).  User¬ 
generated  document  which  contains  information  on  mission  and 
threat,  alternative  concepts,  technology,  and  funding 
implications  of  the  weapon  system  development  (14:Vol 

VII ,52) . 

15.  Office  of  Management  and  Budget  (0MB)  Circular 
A-109.  This  document  is  the  institutional  basis  for  major 
systems  acquisitions  for  all  executive  agencies.  It 
contains  management  principles  for  research,  development, 
and  acquisition. 

16.  Operational  Requirements.  Requirements 
specified  by  the  user  and  support  command  including  the 
Statement  of  Need  (SON),  the  System  Operational  Requirements 
Docximent  (SORD),  and  Depot  Support  Requirements  Document 
(DSRD),  and  scenarios  (14:Vol  VII, 54). 

17.  SON.  Document  generated  by  the  user  which 
identifies  the  initial  mission  need  in  general  operational 
terms.  Includes  a  Parametric  Spreadsheet  which  describes 
the  essential  characteristics  and  quantitative/qualitative 
performance  and  technical  requirements  the  user  deems 
relevant  to  the  mission  success.  Also  includes  a  rational 
and  requirements  change  sheet  which  provides  an  audit  trail 
for  agreed-to  program  changes,  cost  and  performance 
tradeoffs,  growth  plans,  and  supporting  rational  for 
decisions  (14:Vol  VII, 69). 

18.  Tactical  Air  Command  (TAC).  Major  Air  command 
responsible  for  tactical  air  operations  including 
counterair,  air  interdiction,  and  close  air  support. 
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19.  Tactical  Air  Command  Liaison  Officer.  The  TAG 
representative  residing  in  the  SPO. 

20.  User.  End-user  of  the  weapon  system  being 
acquired  (14:Vol  VII, 81). 

Benefits  of  Research 

According  to  General  Randolph,  the  command  needs  to 
change  its  philosophy  on  acquiring  weapon  systems, 
especially  in  today's  environment  of  declining  resources  and 
clear  emphasis  on  technological  solutions  to  our  nation's 
defense  needs  (39).  He  launched  a  campaign  in  late  1989 
that  would  ultimately  share  the  best  practices  found 
anywhere  in  the  AFSC  acquisition  community.  The  draft 
handbook  produced  in  this  process  will  give  the  program 
director  a  set  of  guidelines  for  developing  and  implementing 
a  user-based  interface  strategy  for  refining  the  system 
concept  in  D/V. 

The  ATF  ESC  user-interface  strategy  and  process  can  be 
integrated  into  the  operational  requirements  process  through 
the  APEX  initiative.  Specifically,  it  provides  a  unique  and 
effective  way  to  address  requirements  tradeoffs,  which  is 
critical  to  the  TAF  requirements  process.  Also,  it  fits 
within  the  current  management  guidelines  of  the  TAF 
requirements  process  and  was  easily  incorporated  into  the 
acquisition  process. 

The  ATF  SPO  developed  an  ESC  interface  process  with 
the  user  in  the  D/V  phase  of  their  program.  It  has  been 
tested  and  accepted  by  the  SPO  and  the  user.  This  process 
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can  be  tailored  to  other  major  programs  within  the  current 
guidelines  of  Air  Force  acquisition  policy,  documentation, 
and  the  requirements  process,  unlike  the  incompatible  user- 
based  acquisition  approach  the  APEX  committee  recommended. 
Finally,  the  Air  Force  Inspector  General  (IG)  reported  that 
the  ATF  ESC  strategy  and  user-interface  process  is 
innovative  and  effective  (7). 

Chapter  Summary 

A  smooth,  systematic,  and  effective  process  to  make 
performance  tradeoffs  in  the  D/V  phase  of  the  acquisition 
process  is  long  overdue.  The  APEX  committee  was  formed 
because  the  current  process  of  developing  acquisition 
strategies  lacks  a  consistent,  systematic  approach,  is  not 
responsive  to  the  user,  does  not  comprehensively  address 
program  risks,  and  results  in  a  lack  of  commitment  (15;  39). 
However,  most  of  the  initiatives  related  to  the  APEX  study 
have  ignored  the  interface  between  AFSC  and  the  user.  One 
initiative.  Early  Acquisition  Strategy,  placed  the  user  in 
charge  of  an  acquisition  team  in  the  CE/D  phase.  This 
approach  has  not  been  approved  by  the  Air  Force.  It  may 
change  Air  Force  acquisition  policies  and  procedures,  and 
alter  the  TAF  requirements  process  roles  and 
responsibilities.  The  ATF  SPO  has  developed  its  own  user- 
based  interface  process  which  is  early,  structured,  and 
continuous,  and  has  been  tested  and  accepted  by  the  ATF  SPO 
and  the  user.  More  importantly,  this  ESC  interface  process 
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does  fit  within  the  guidelines  of  Air  Force  policy.  This 
study  will  identify  activities  that  can  improve  the 
performance  tradeoffs  portion  of  the  requirements  process  by 
evaluating  the  ATF  ESC  user-interface  process. 

Overview  of  Thesis 

The  purpose  of  Chapter  I  was  to  introduce  the  study. 

The  background,  general  issue,  research  problem,  investiga¬ 
tive  questions,  scope  and  limitations,  definitions,  justi¬ 
fication,  and  benefits  of  research  were  discussed. 

In  Chapter  II,  the  methodology  for  researching  and 
answering  the  research  problem  is  discussed.  This  section 
includes  an  introduction  of  the  research  design,  thesis 
sponsorship,  and  the  research  methodology  for  answering  the 
investigative  questions. 

Chapter  III  outlines  the  major  highlights  of  the 
literature  review  in  preparation  for  this  study. 

The  research  findings  are  summarized  in  Chapter  IV,  and 
the  conclusions  and  recommendations  are  summarized  in 
Chapter  V. 
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II .  Methodology 


Introduction 

The  handbook  conveyed,  in  part,  the  perceptions  of  the 
APEX  team,  user,  ATF  SPO,  and  contractors  associated  with 
the  ATF  ESC  user-interface  process.  To  gather  these 
perceptions,  an  orderly  approach,  or  research  method,  to 
collect  the  research  data  was  first  developed. 

The  research  method  used  in  this  study  consisted  of 
unstructured  personal  interviews  and  a  literature  review  to 
identify  the  management  and  engineering  techniques 
associated  with  the  ATF  ESC  process.  The  study  criteria  as 
well  as  the  general  and  specific  problems  were  identified 
and  validated,  using  the  survey  method,  by  the  APEX 
evaluation  of  the  current  acquisition  process.  The 
unstructured  personal  interviews  allowed  for  a  more  detailed 
and  in-depth  study  of  the  ATF  ESC  process,  and  they 
identified  possible  improvements  to  the  ATF  approach. 

Thesis  Sponsorship.  During  the  initial  phases  of 
defining  and  refining  the  research  topic,  informal 
discussions  were  conducted  with  the  ATF  TAC  Liaison  Officer, 
the  ATF  Director  of  Projects,  the  Chief,  Analysis  Branch, 
Directorate  of  Projects,  Advanced  Tactical  Fighter  System 
Program  Office,  and  the  Chief,  Plans  and  Strategies  Branch, 
Directorate  of  Projects,  Advanced  Tactical  Fighter  SPO. 

After  these  discussions,  they  concluded  that  a  study  of  the 
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ATF  ESC  user-interface  model  and  the  development  of  a 
handbook  for  this  process  would  be  useful  to  other  Air  Force 
program  managers. 

Justification  of  Approach 

The  purpose  of  this  research  was  to  develop  a  handbook 
that  documented  a  "best  practice"  for  involving  the  user  in 
the  acquisition  process.  Eventually,  this  handbook  could  be 
used  by  other  program  directors  and  SPO  personnel  for 
developing  and  implementing  an  effective  and  proven  SPO 
user-based  interface  strategy. 

The  study  used  results  from  the  APEX  survey  and  the 
ATF  ESC  user-interface  process  management  and  engineering 
techniques  to  organize  and  develop  the  handbook.  The  study 
also  identified  potential  improvements  to  the  ATF  ESC 
process.  Therefore,  in  order  to  accomplish  this  second 
objective,  the  perceptions  of  four  groups  of  people  had  to 
be  collected:  APEX  team,  ATF  SPO,  user,  and  contractor. 

The  indepen<fent  variables  involved  in  the  APEX  study 
were  categorized  as  attitudes  and  opinions.  Emory  points 
out  that  attitudes  and  opinions  can  only  be  measured  through 
the  survey  method  (28:158).  "Surveying  can  be  carried  out 
by  face-to-face  interviewing,  by  telephone,  by  mail  or  a 
combination  of  these"  (28:159).  Therefore,  the  APEX  team's 
survey  technique  employed  all  three  methods. 

In  order  to  obtain  the  appropriate  information  for  this 
handbook,  a  literature  search  of  DoD  studies  on  improving 
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the  acquisition  process  and  ATF  SPO  documentation  and  a 
consensus  survey,  in  the  form  of  interviews,  were  required. 
According  to  Emory,  this  information  could  be  obtained  by 
the  use  of  a  literature  search  or  the  survey  method 
(28:158) . 

Identification  of  Population 

It  was  necessary  to  identify  individuals  in  the  ATF 
SPO,  the  User,  civilian  contractors,  and  the  APEX  team  who 
were  familiar  with  the  ATF  ESC  strategy  and  user-interface 
process.  Unstructured  personal  and  telephone  interviews 
with  ATF  SPO  and  APEX  team  personnel  aided  in  the 
identification  of  the  appropriate  population.  Specifically, 
the  ATF  SPO  TAG  Liaison  Officer  (TACLO),  the  Chief  of  the 
ATF  SPO  Tactical  Requirements  Branch,  and  the  Chief  of  the 
ATF  Strategies  Branch,  assisted  in  this  process. 

The  ATF  SPO  population  of  interest  included  the  ATF 
director,  the  chief  engineer,  the  deputy  director,  the 
assistant  director,  and  the  directors  and  branch  chiefs  of 
Logistics,  Projects,  and  Engineering.  Additionally,  all  the 
people  directly  involved  with  the  ATP  ESC  strategy  and 
process  in  the  ATF  SPO  were  also  interviewed. 

The  user  population  of  interest  includes  the  ATF  TAC 
Liaison  Officer,  the  ATF  Deputy  Program  Manager  of 
Logistics,  the  ATP  ATC  representative,  and  all  other  user 
personnel  collocated  in  the  ATF  SPO.  In  addition,  user 


personnel  fron  TAC/DRB,  who  were  directly  involved  with  the 
ATP  approach,  were  interviewed. 

The  contractor  population  was  defined  as  the  focal 
point  for  the  various  activities  associated  with  the  ATP 
approach.  The  ATP  contractors  included  the  four  prime 
contractors  who  participated  in  the  ATP  D/V  phase.  These 
people  were  chosen  because  of  their  familiarity  with  the 
techniques  used  in  the  ATP  user-interface  process. 

Moreover,  they  participate  in  the  performance  trades 
process . 

Pinal ly,  the  APEX  team  population  included  APSC,  SPO, 
and  TAP-user  personnel  that  were  part  of  the  user  team. 

This  team  actually  interviewed  the  program  managers  and 
functional  experts  during  the  evaluation. 

The  APEX  Evaluation 

Hundreds  of  interviews  were  conducted  by  APEX  team 
members  to  identify  no-value-added  tasks  associated  with  the 
acquisition  of  weapon  systems  which  could  or  should  be 
deleted  or  improved  (15).  Program  directors,  project 
managers,  industry,  users,  functional  organization 
personnel,  and  acquisition  executives  from  qther  services 
were  interviewed  by  the  team  interviewees  (15).  In  addition 
to  the  perceived  problems  and  problem  areas,  more  than  300 
improvement  ideas  were  identified. 

The  APEX  team  concluded,  in  part,  that  SPOs  needed  to 
develop  a  better  interface  process  with  their  participants. 
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including  the  user,  particularly  in  the  early  phases  of  the 
acquisition  process  (15).  Also,  the  study  identified  a 
number  of  concern  areas  associated  with  the  requirements 
generation  process  which  every  SPO-user  team-building 
strategy  should  consider.  They  were  (16): 

Good  Cooperation  and  Exchange  of  Ideas.  The  program 
director  and  his  user  counterpart  must  create  an  environment 
of  openness  and  trust.  They  must  be  committed  to  the 
establishment  of  an  effective  SPO-user  team  process.  They 
must  also  ensure  that  the  personnel  responsible  for 
implementing  the  user-interface  strategy  have  the  necessary 
guidance.  The  user  must  be  integrated* into  the  acquisition 
strategy,  and  be  involved  in  every  aspect  of  the  program. 

Disciplined  Requirements  Generation.  The  user  believes 
the  requirements  definition  process  should  be  separated  from 
the  Planning  Programming  and  Budgeting  System  (PPBS) 
process.  The  requirements  process  is  set  up  to  make 
informed  cost-schedule-perf ormance-tradeof f s  at  critical 
points  during  the  acquisition  process.  Whereas,  the  PPBS 
typically  is  not  set  up  to  make  informed  affordability 
assessments  and  resource  allocation  decisions  on  defense 
acquisition  programs.  Moreover,  SPO  analysis  techniques  do 
not  adequately  reflect  cost  and  schedule  constraints.  Also, 
the  user  feels  that  AFSC  needs  to  commit  to  Preplanned 
Product  Improvement  (P3I)  programs  earlier  in  the 
acquisition  cycle. 
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Team  Approach.  The  SPO  user-interface  strategy  should 
consider  a  team  acquisition  strategy  throughout  the  life  of 
the  program.  The  user  should  be  involved  in  the  day-to-day 
decisions.  The  team  should  operate  with  complete  candor. 
Finally,  the  user  needs  to  be  better  educated  in  the 
acquisition  process. 

Mission  Area  Anal vsis/Cost  of  Operational  Effectiveness 
Analysis  (MAA/COEA).  Historically,  AFSC  has  not  committed 
the  necessary  resources  to  execute  this  analysis.  Moreover, 
the  user  has  been  excluded  from  this  analysis. 

Concept  DIR  Studies.  This  concern  is  not  applicable  to 
this  study. 

Approach  to  Major  and  Non-Maior  Programs.  This  concern 
is  not  applicable  to  this  study. 

Exclusion  from  Acquisition  Process.  Typically,  when 
the  operational  needs  are  identified,  the  user  does  not 
remain  a  responsible  and  useful  partner  in  the  development 
and  acquisition  process.  .  Ideal lx,  the  user  must  be  involved 
in  every  program  action  to  ensure  the  TAP  is  provided  with 
the  best  operational  capabilities. 

Improved  Synergism  of  Documentation.  A  strategic  plan 
needs  to  be  developed  up  front  to  synergise  all  applicable 
program  documentation.  This  includes  clear  system 
definition,  a  threat  baseline,  and  standard  simulations.  It 
is  critical  that  the  system  specification  must  be  traceable 
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back  to  the  system  requirements  and  the  Operational 
Requirements  Document  (ORD)  through  the  trade  studies. 

Also,  the  ORD  should  be  updated  after  every  major  trade  or 
program  action  rather  than  at  every  major  milestone.  All 
this  is  key  to  having  accurate  and  timely  information 
available  (Whenever  it  is  needed. 

Handbook 

Typically,  AFSC  program  managers  and  SPO  personnel  do 
not  have  proper  guidance,  experience,  or  knowledge,  to 
integrate  the  user  into  the  acquisition  process.  Therefore, 
the  ATF'based  ESC  interface  process  handbook  was  written  for 
all  AFSC  program  directors  and  managers.  The  handbook  could 
also  be  useful  to  SPO  and  user  personnel  who  are  responsible 
for  planning,  implementing  and  executing  their  program 
manager's  user-based  interface  process. 

Literature  Review 

The  literature  review  provided  the  initial  background 
information  on  the  ATF  strategy  and  the  corresponding  user- 
based  process.  Specific  engineering  and  management 
techniques  developed  by  the  ATF  SPO  and  the  user  which  would 
improve  communication  and,  therefore,  the  contributions  were 
initially  identified  and  documented  in  the  handbook  under 
the  APEX  concern  areas.  According  to  Business  Research 
Methods,  by  Emory,  valuable  background  information  is 
acquired  through  a  literature  review  (28:135-155). 
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Interview  Approach 

The  interviews  allowed  for  more  detailed  and  in-depth 
study  of  the  subject,  and  they  obtained  essential  informa¬ 
tion  that  was  needed  to  write  the  handbook.  Also,  through 
the  interviews,  it  was  possible  to  identify  possible 
improvements  to  the  ATF  ESC  process. 

The  unstructured  interview  was  used  in  this  study 
because  of  its  flexibility.  Even  though  a  statistical 
analysis  was  not  done,  this  type  of  interview  enabled  verbal 
communication  between  the  researcher  and  the  person  being 
interviewed.  Another  benefit  of  the  unstructured  interview 
was  that  verbal  communication  allowed  mis\inderstandings  or 
ambiguities  to  be  resolved  during  the  interviews.  The 
opportunity  to  clear  up  ambiguities  was  highly  desirable 
because  of  the  unique  nature  of  this  study. 

The  most  serious  shortcoming  of  any  interview  is  the 
amount  of  time  required  to  contact  and  interview  the 
necessary  people.  Thus,  because  of  time  constraints,  the 
number  of  people  who  can  be  contacted  and  interviewed  is 
limited.  This  time  constraint  was  further  complicated  by 
the  fact  that  most  of  the  user  and  APEX  personnel  were  not 
located  at  ASD.  In  some  instances,  there  was  no  alternative 
but  to  do  a  telephone  interview  and  coordinate  the  results 
with  the  interviewee  after  they  were  drafted. 

Interview  Construction  and  Administration.  The 
investigative  questions  were  developed  to  enable  the  study 
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to  not  only  docxunent  the  ATF  ESC  process,  but  possibly 
improve  it.  In  order  to  answer  the  six  investigative 
questions  and  obtain  the  information  for  the  handbook,  a 
subgroup  of  the  sample  was  interviewed  for  each 
investigative  question. 

Investigative  Question  One.  What  factors  caused 
the  ATF  SPO  to  develop  the  ESC  interface  process  process 
with  the  User?  The  ATF  SPO  Director,  the  ATP  Deputy  Program 
Director,  the  Assistant  ATF  Program  Director,  the  Deputy 
Chief  of  Staff,  Tactical  Air  Conmiand,  and  the  APEX  committee 
chairperson  were  interviewed  to  answer  Question  1.  Several 
of  these  people  are  located  in  Washington  D.  C.,  so  TDY 
travel  was  required. 

Investigative  Question  Two.  What  changes  were 

I 

required  in  ihe  ATF  SPO's  acquisition  strategy  to  support 
the  ATF  ESC  interface  process  with  the  User?  The  ATF 
Program  Director,  the  ATP  deputy  program  director,  the 
Assistant  ATF  Program  Director,  the  ATF  Director  of 
Projects >  the  Chief,  Plans  and  Strategies  Division,  ATF  SPO, 
HQ  USAF,  and  HQ  APSC  were  interviewed  to  answer  Question  2, 
Question  5,  and  Question  6. 

Investigative  Question  Three.  What  engineering 
and  management  techniques  were  developed  by  the  ATF  SPO  for 
the  ESC  interface  process  with  the  User?  The  whole  sample 
was  interviewed  to  answer  Question  3. 
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Investigative  Question  Four.  How  can  these 
engineering  and  management  techniques  be  generalized  and 
applied  to  other  major  Air  Prrce  acquisition  programs? 

The  whole  sample  was  interviewed  to  answer  Question  4. 

Investigative  Question  Five.  Into  which 
acquisition  phase  should  the  program  director  implement  the 
ATP  interface  process  with  the  User?  The  ATP  Program 
Director,  the  ATP  deputy  program  director,  the  Assistant  ATP 
Program  Director,  the  ATP  Director  of  Projects,  the  Chief, 
Plans  and  Strategies  Division,  ATP  SPO,  HQ  USAF,  and  HQ  AFSC 
were  interviewed  to  answer  Question  5. 

Investigative  Question  Six.  What  approval  cycle 
is  required  to  implement  the  ATP  ESC  interface  process  with 
the  User?  The  ATP  Program  Director,  the  ATP  deputy  program 
director,  the  Assistant  ATP  Program  Director,  the  ATP 
Director  of  Projects,  the  Chief,  Plans  and  Strategies 
Division,  ATP  SPO,  HQ  USAF,  and  HQ  APSC  were  interviewed  to 
answer  Question  6. 

Depending  on  the  progress  of  the  handbook,  additional 
research  and  interviews  may  be  required  to  understand  the 
process  of  validating  a  handbook.  A  sample  of  all  these 
people,  including  Major  Stibravy,  Dr.  Shane,  and 
Captain  Daley  (AFIT  90S  graduate),  will  be  interviewed  to 
answer  Question  4. 
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Chapter  Summary 

The  purpose  of  Chapter  II  was  to  present  the  research 
methodology  for  answering  the  investigative  questions  and 
addressing  the  specific  APEX  concern  areas.  The 
population  was  identified,  including  the  validation  method. 
The  measurement  instruments,  literature  review,  and 
interviews,  were  identified,  and  the  specific  procedures  for 
conducting  the  interviews  were  discussed.  The  literature 
review  will  be  discussed  in  more  detail  in  Chapter  III. 
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Ill .  Literature  Review 


Introduction 

The  purpose  of  this  chapter  is  to  examine  literature 
related  to  the  acquisition  process  and  the  TAP  requirements 
process  within  industry  as  well  as  the  government  to  provide 
a  better  understanding  of  weapon  system  acquisitions.  In 
addition,  this  chapter  began  docximenting  some  of  the  ATP  ESC 
user-interface  concepts.  The  first  section  provides  a  brief 
overview  of  the  Acquisition  process,  TAP  requirements 
process,  and  TAP  requirements  management  responsibility. 
Specific  relationships  were  also  discussed  under  management 
responsibilities.  This  included:  TAC/TAP  relationship, 
program  objectives  memorandxim  (POM) /budget  submission 

I 

estimates  (BES)  involvement,  APSC,  APLC,  and  contractor 
relationships.  The  second  section  reviews  and  outlines  some 
of  the  concepts  associated  with  the  ATP  ESC  user  interface 
process.  Pinal ly,  the  third  section  reviews  three  studies 
conducted  within  APSC  that  apply  specifically  to  the  topic 
of  research. 

Scope  of  the  Research  Topic 

Research  on  the  subject  of  SPO-user  relationships 
during  the  requirements  process  included  a  search  of  the 
literature  listed  in  DTIC  and  professional  journals  within 
the  libraries  of  APIT  and  Wright  State  University.  In 
addition,  knowledgeable  people  within  the  area  were  sought 
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and  asked  about  published  or  unpublished  literature  that 
would  be  applicable  to  this  study.  These  sources  included 
the  ATP  SPO,  ATP  TACSO,  ATP  Director  of  Projects,  TAC/DR, 
and  various  personnel  from  the  APEX  team. 

Discussion  of  the  Literature 

General  Randolph,  former  commander  of  Air  Porce  Systems 
Command  (APSC),  spearheaded  an  initiative  in  1989  that 
sought  out  ways  to  improve  the  current  acquisition  process. 
Major  changes  were  required  because  of  today's  environment 
of  declining  resources  and  clear  emphasis  on  technological 
solutions  to  our  nation's  defense  needs  (15;  39).  This 
initiative  later  became  known  as  APEX.  The  mission  of  APEX 
was  to  create  a  revolutionary  team  acquisition  process  that 
destroyed  barriers,  inspired  opportimity  for  improvement, 
and  increased  responsiveness  to  the  user  (15). 

During  the  APEX  team's  data  collection  phase  which 
consisted  of  surveying  program  managers  and  functional 
experts  across  the  Command,  recurring  inputs  on  perceived 
acquisition  problems  centered  on  ways  to  improve  the 
relationship  between  APSC  and  the  user  (15).  One  of  the 
most  promising  recommendations  from  the  APEX  team's  initial 
study  was  the  Early  Acquisition  Strategy,  which  placed  the 
user  instead  of  APSC  in  charge  of  the  Concept  Exploration/ 
Definition  phase. 

The  ATP  SPO  developed  a  similar  process  for  improving 
the  relationship  with  the  User,  the  Early,  Structured,  and 
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Continuous  (ESC)  user-interface  process,  in  the 
Demonstration/Validation  phase  of  the  program.  The  goal  of 
the  process  was  to  make  the  User  an  accountable  team  member, 
and  to  educate  the  User  to  ensure  the  requirements  were 
stablized  as  early  as  possible  in  the  acquisition  process. 
This  process  improved  the  way  in  which  the 
SPO/contractor/user  team  refined  the  initial  requirements 
through  the  trade  study  process. 

Major  Systems  Acquisition 

Most  new  development  programs  go  through  a  sequence  of 
key  program  decision  milestones  and  activities  known  as  the 
acquisition  process  or  system  acquisition  life  cycle.  The 
system  acquisition  life  cycle  is  essentially  a  logical  flow 
of  activity  representing  an  orderly  progression  from  concept 
formulation  to  final  operational  deployment  and  support 
(37:258).  Outputs  of  each  pha^e  constitute  a  definite  and 
documented  baseline  for  entry  into  the  next  phase.  The  life 
cycle  also  represents  an  incremental  commitment  of 
resources.  As  the  system  evolves,  each  decision  milestone 
is  directed  to  commitment  of  increased  resources  predicted 
on  achievement  of  well-defined  objectives.  The  milestone 
objectives  are  designated  to  reduce  the  risks  (cost, 
schedule,  and  technical)  inherent  in  a  new  acquisition 
through  demonstrated  performance,  thus  enabling  acquisition 
programs  to  progress  to  completion  or  be  terminated 


Definition  of  Major  System.  The  SECDEP  designates 
those  programs  that  are  to  be  managed  as  major  systems. 
Normally,  this  is  done  when  the  system  is  first  authorized 
(MNS  is  approved) .  The  decision  to  designate  those  systems 
as  major  may,  after  consultation  with  the  USAF,  be  based  on 
the  following  (43: Chapter  5,1): 

1.  Development  risk  or  urgency  of  need. 

2.  Joint  acquisition  of  a  system  by  the  DoD  and 
representatives  of  another  nation,  or  by  two  or  more  DoD 
components . 

3.  The  estimated  Research  Development  Test  and 
Acquisition  (RDT&E) ,  procurement,  operations  and  support 
costs  (rule  of  thumb  is  $200M  in  RDT&E  or  $1B  in  procurement 
funds) . 

4.  Significant  Congressional  interest. 

Initial  SECDEF  Approval .  Systems  acquisitions  are 
based  on  operational  needs  identified  through  continuing 
analysis  of  missions.  The  Statement  of  Operational  Need 
(SON)  or  Mission  Need  Statement  (MNS)  must  specify  the  need 
narrowly  enough  to  allow  a  reasonable  probability  that  a 
single  system  can  correct  the  deficiency  without  identifying 
specific  hardware  solutions  (43:Chapter  5,3). 

Each  major  system  acquisition  program  requires  a  MNS  to 
be  reviewed  by  OSD  in  the  POM  review  before  the  program  is 
included  in  the  DoD  budget  submission.  When  the  MNS  is  sent 
to  OSD,  they  must  contain  recommendations  and  rationale  for 
designating  the  program  a  major  program. 
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The  SECDEF  approves  the  new  start  via  an  Acquisition 
Decision  Memorandum,  which  authorizes  the  start  of  the 
acquisition  process  and  entry  into  the  Concept  Exploration/ 
Definition  phase. 

Following  SECDEF  approval,  HQ  USAF  provides  formal 
direction  to  the  implementing  and  participating  commands  by 
issuing  a  Program  Management  Direction  (PMD).  The  PMD  is 
used  throughout  the  entire  acquisition  life  cycle  to  state 
requirements  and  request  studies  as  well  as  initiate, 
approve,  transfer,  modify,  or  terminate  programs. 

TAF  Requirements  Process 

The  ATF  ESC  interface  process  was  implemented  without 
changing  the  TAF  requirements  process  or  the  TAF 
requirements  process  management  responsibility.  These 
processes  provide  the  means  to  identify  operational 
deficiencies,  state  operational  needs,  and  initiate 
development  of  new  (or  improvements  to  existing)  systems  and 
equipment . 

Deficiencies  in  force  capabilities  may  be  identified 
through  Mission  Area  Analysis  (MAA).  Operational 
deficiencies  which  cannot  be  satisfied/corrected  within  the 
command's  authority  must  be  identified,  quantified,  and 
submitted  to  HQ  USAF  (37:301).  The  statement  of  deficiency 
or  need  is  done  by  the  Statement  of  Operational  Need  (SON) 
in  accordance  with  AFR  57-1,  Operational  Needs . 
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Requirements ,  and  Concepts ♦  At  this  point,  TAC/DR  becomes 
the  responsible  TAP  agency  for  the  remainder  of  the  TAP 
requirements  process. 

Poliowing  SON  validation,  or  SECDEP  approval  for  major 
systems,  a  PMD  is  issued  by  HQ  USAP.  In  addition,  TAC 
DCS/Plans  provides  Preliminary  System  Operational  Concepts 
(PSOCs)  and  System  Operational  Concepts  (SOCs)  on  candidate 
solutions  to  SONS  (43:Chapter  5,  3-6).  These  documents 
describe  the  intended  purpose,  employment,  deployment,  and 
support  of  a  system. 

Requirements  Management 

TAC/DR  maintains  cVse  contact  throughout  the  entire 
acquisition  process  from  program  initiation  to  production 
and  deployment  and  operational  support  (43).  , 

To  provide  single,  cohesive,  and  responsive  focal  point 
for  TAP  modernization,  the  Deputy  Chief  of  Staff  (DCS) 
Requirements  was  established  (43). 

TAC/DR  is  subdivided  into  six  directorates  and  three 
major  Special  Management  Organizations  (SMOs)  to  work  and 
manage  programs.  To  assist  DR  in  its  management  function, 
the  Liaison  office  was  formed.  DR  has  eight  liaison  offices 
located  at  selected  Air  Porce  activities,  primarily  APSC 
divisions.  They  serve  as  the  TAP  representatives  and  play  a 
major  role  in  their  management  of  TAP  requirements. 

TAP/TAC  Relationship.  TAC/DR  is  designated  the 
requirements  spokesman  for  the  TAP  (TAC,  USAPE,  PACAP) 
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throughout  the  development  and  acquisition  process  (43). 

They  perform  the  administrative  functions  of  coordination, 
justifying,  and  presenting  TAF  requirements. 

POM/BES  Involvement.  Another  key  aspect  of  TAC's 
involvement  in  the  requirements  process  is  their  attempt  to 
influence  the  formulation  of  the  POM/BES  to  ensure  that 
tap's  most  urgent  needs  are  funded  (43). 

TAG  POM  working  groups,  with  DR  representatives  and/or 
chairmen,  are  critical  elements  in  the  efficient  and 
effective  production  of  a  prioritized  listing  of  TAF  RD&A 
programs  for  submission  to  Air  Staff  for  their  use  during 
POM  formulation  (43). 

Testing  Responsibilities.  Testing  is  another  vital 
ingredient  in  a  successful  acquisition  program-  There  are 
two  types  of  testing  associated  with  acquisition: 

Development  Test  and  Evaluation  (DT&E)  and  Operational  Test 
and  Evaluation  (OT&E) .  DTSE  is  normally  performed  by  AFSC. 
TAC/DR  is  the  responsible  staff  agency  for  all  OTSE 
conducted  within  the  command. 

Relationship  with  AFSC.  TAC/DR  is  deeply  involved  with 
AFSC  because  they  are  responsible  for  the  development  and 
acquisition  of  weapon  systems  and  support  equipment  that  are 
responsive  to  the  needs  of  the  TAF  (43: Chapter  5,17).  It  is 
important  that  TAC/DR  and  AFSC  work  closely  throughout  the 
acquisition  process. 
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Relationship  with  AFLC.  TAC/DR  is  also  involved  with 
AFLC  on  programs  which  modify  inventory  systems  and  support 
equipment  (43: Chapter  5,18). 

Relationship  with  Contractor.  TAC/DR  has  frequent 
contacts  with  contractor  representatives,  and  it  is 
important  that  they  understand  what  the  Air  Force  expects 
that  relationship  to  be  (43:Chapter  5,19). 

Overview  of  ATF  Program 

The  ATF  is  being  developed  as  an  air  superiority 
fighter  to  dominate  the  threat  in  2000  and  beyond  with  a 
primary  design  mission  of  offensive  counterair  (13). 
Developing  a  system  with  improved  performance 
characteristics  such  as  supersonic  persistency, 
high  maneuverability,  and  better  combat  radius,  with  goals 
for  a  50,000  pound  aircraft  at  $35  million  average  unit 
flyaway  cost  (base  year  '85  dollars), was  a  challenging  task 
requiring  optimization  of  numerous  variables  which  must  be 
treated  in  an  integrated  analysis  (13).  The  program  is 
currently  in  a  competitive  D/V  phase  and  will  enter  FSD  in 
mid-1991.  Production  options  are  scheduled  to  be  exercised 
beginning  in  1996  with  TAF  receiving  aircraft  beginning  in 
1999  (13). 

ATF  ESC  User-Interface  Process 

User-Interface  Philosophy.  This  section  discusses  some 
of  the  management  and  engineering  techniques  developed  by 
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the  ATF  SPO  that  an  acquisition  program  can  use  in  the  D/V 

phase  to  help  ensure  the  operational  requirements  are  met. 

The  user  was  involved  in  every  aspect  of  the  program  to 

ensure  operational  requirements  become  an  integral  part  of 

the  weapon  system  design  process.  Representatives  from  the 

TAP,  AFLC,  and  ATC,  for  the  ATF,  were  collocated  in  the 

program  office  and  treated  as  members  of  the  SPO.  There 

were  no  formal  agreements  between  the  SPO  and  the  User, 

since  senior  Air  Force  management  supported  the  approach. 

The  ATF  SPO  established  teams  of  O&S  and  MPTS  experts  within 

the  SPO  to  ensure  these  requirements  were  integrated  into 

the  system  design.  For  example,  the  ATF  user-interface 

approach  for  MPTS  was  as  follows: 

In  the  past  MPTS  has  been  an  analysis  activity  that 
takes  the  results  of  a  design  process  to  find  an  MPTS 
answer  to  a  fixed  design.  MPTS  analysis  must  be 
applied  through  the  design  process  so  that  the  result 
is  a  balanced  weapon  design  that  includes  an  optimized 
MPTS  solution.  (13) 

The  user  attended  working  groups.  Technical  Interchange 
Meetings  (TIM),  program  reviews,  and  briefings  at  all 
levels.  The  user  participated  in  MAA/COEAs  and  ATF- 
alternative  studies.  The  program  director  briefed  the  TAG 
commander  every  six  months  on  the  status  of  the  weapon 
system  design.  He  also  effectively  used  a  direct  line  to 
the  TAG  Gommander,  through  TAG/DRB,  which  was  exercised  even 
more  frequently  due  to  the  priority  of  the  program  and  the 
political  sensitivity. 
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ATF  Threat  Baseline.  The  program  director  ensured  the 
user  was  involved  in  the  process  of  baselining  the  threat 
description.  A  standardized  threat  description  was  critical 
for  several  reasons.  First,  the  competing  contractors  had 
to  be  evaluated  from  a  common  baseline.  Second,  a  standard, 
coordinated  threat  baseline  helped  to  defend  the  program  in 
the  annual  program  reviews  and  alternative  fighter  studies. 
The  ATF  SPO  went  to  great  lengths  to  accomplish  the 
necessary  coordination  as  indicated  in  a  letter  from  the  Air 
Force  Chief  of  Staff: 

The  Air  Force  ATF  decision-making  community  from  the 
SPO  to  the  Secretary  needs  a  known,  documented, 
authentic  baseline  threat  and  model.  For  example,  the 
contractors  need  a  coordinated,  definitive  baseline 
threat  from  AF/IN  (DIA  certified)  and  AF/SA  needs  to 
control  and  operate  a  baseline  model  so  as  to  make  the 
contractor  variants  either  conform  or  be  visibly  not  in 
conformance.  (36) 

The  ATF  SPO  established  a  Threat  Working  Group  (TWO)  with  a 
charter  that  ensured  that:  (a)  the  baseline  was 
established,  (b)  the  contractor  conformed  to  the  threat 
description,  and  (c)  the  baseline  was  periodically  updated. 
Participants  from  every  applicable  Air  Force  organization 
participated  in  this  group. 

Contractor  Trade  Studies.  The  trade  study  process  was 

critical  in  refining  the  requirements,  reducing  risk,  and 

ensuring  the  data  was  traceable  and  justifiable.  The 

process  can  be  described  as  follows: 

This  process  includes  acquiring  sufficient  data  to  make 
requirements  decisions  and  maturing  technologies  with 
which  to  enter  FSD,  both  risk  reduction  activities. 
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This  is  an  iterative  process  in  which  the  SPO  provided 
the  contractors  a  system  requirement  against  which  the 
contractors  are  applying  trade  studies  and  demonstra¬ 
tion  tools,  returning  recommendations  for  changes  to 
the  requirement  for  better  system  optimization.  These 
requirements  recommendations  were  analyzed  by  the  Air 
Force.  Accepted  recommendations  are  documented  in  the 
system  specification  and  returned  to  the  contractor 
as  new  requirements.  This  cycle  was  repeated  annually 
during  the  0/V  phase.  (1;  34;  40) 

The  process  results  in  a  match  of  requirements  and  available 
technologies  to  baseline  an  executable  program  at  FSD 
(1).  This  led  to  an  unprecedented  technical  understand¬ 
ing  of  the  requirements  and  a  complete  traceability  to  the 
original  requirements.  Cost  was  factored  into  requirements 
trades.  This  process  identified  the  requirements  that  had 
to  be  flight  tested  in  D/V.  The  program  entered  FSD  with 
low  to  moderate  program  risk  in  all  areas.  The  TAC 
Commander  reviewed  the  status  of  the  contractor  trade 
studies  on  an  annual  basis.  This  review  required 
unparalleled  teamwork  between  the  SPO,  user,  contractor,  and 
other  government  organizations.  In  short,  the  user  had  a 
willingness  to  adjust  his  requirements  based  on  the 
contractor  trade  studies  which  demonstrated  technical 
feasibility. 

System  Design  Concept.  The  ATF  was  designed  primarily 
for  a  single  mission  requirement-offensive  counterair.  The 
design  philosophy  stressed  utilization  of  state-of-the-art 
technology  and  the  descriminate  application  and  tailoring  of 
military  standards  and  specifications  (37:293).  Another 
objective  of  ATP  D/V  was  to  force  integration  of 
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supportability  and  MPTS  requirements  into  the  weapon  system 
design  and  deliver  a  plan  that  ensured  these  requirements 
were  met  (13).  Normally,  supportability  and  MPTS  take  the 
results  of  the  design  process  to  find  a  supportability  and 
MPTS  answer  to  a  fixed  design  (13).  The  ATF  SPO  ensured 
this  design  concept  was  adhered  to  with  the  following 
process : 

The  government  process  includes  assembling  a  team  with 
MPTS  experts  within  the  SPO  to  work  MPTS  requirements. 
This  MPTS  team  can  be  extended  to  Training  Planning 
Teams  (TPT),  expanding  the  scope  of  TPTs  and  providing 
a  forum  for  MPTS  discussions.  (13) 

The  SPO  had  the  same  philosophy  and  approach  in  considering 

supportability  requirements.  This  review  did  not  address 

the  corresponding  contractor  process  and  the  relationship 

between  the  government  and  contractor  processes. 

Budget  Model .  Affordability  was  a  major  program 

driver  in  the  annual  review  process  and  forced  discipline  in 

« 

the  system.  A  relational,  detailed  budget  model  was 
developed  to  validate  the  contractor  data  in  relative  terms 
and  cost  the  proposed  configurations  as  they  were  developed. 
This  model  was  developed  from  Air  Force  and  contractor  data 
(23;  35). 

SORD.  The  System  Operational  Requirements 
Document  (SORD)  was  updated  annually  instead  of  at  major 
milestones.  This  ensured  that  the  system  specification  and 
SORD  were  consistent. 
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Working  Groups.  Working  Groups  were  established 
for  weapons  integration,  avionics,  cockpit,  training,  sup- 
portability,  safety,  and  test  and  evaluation.  These  groups 
consisted  of  representatives  from  the  government  and  con¬ 
tractor  organizations,  chaired  by  the  ATF  SPO  management, 
and  met  as  required  to  assist  in  the  development  of  an  in¬ 
tegrated  weapon  system  design  (3;  4;  5;  11;  12).  For 
example,  the  user  inputs  to  the  Threat  Working  Group  helped 
in  validating  astandardized  threat  baseline  for  use  by  the 
contractors  and  TAG.  Pilots'  participation  in  the  Cockpit 
Working  Group  improved  understanding  of  the  man-machine 
interface  (42) 

Weapons  Integration.  Munitions  System  Division 
(MSD)  established  an  operating  location  within  the  ATF  SPO 
to  ensure  a  balanced  weapon  system  design.  A  Memorandum  of 
Agreement  specified  the  required  support  (8).  The 
MSD  representative  was  critical  in  providing  baseline  wea¬ 
pons  data  to  ensure  total  weapon  system  integration,  and 
ensured  armament  program  cost,  schedule,  technical ,  and 
program  direction  was  consistent  with  the  ATF  program.  The 
ATF  SPO  prepared  a  Weapons  Integration  Plan  which  was  a 
major  tool  of  the  Armament  Integration  Working  Group,  and 
ensured  the  ATF  design  was  a  weaponized  design. 

Engine .  The  engine  program  was  a  separate  program  from 
the  weapon  system.  However,  the  ATF  SPO  controlled  the 
engine  program  which  is  normally  managed  by  Deputy  for  Pro- 
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pulsion.  Aeronautical  Systems  Division,  ASD/YZ  in  other  air¬ 
craft  programs.  These  personnel  were  located  in  the  ATF  SPO 
and  reported  directly  to  the  Program  Director.  This  ap¬ 
proach  gave  the  Program  Director  complete  control  of  the  D/V 
process . 

Avionics .  The  ATP  SPO  controlled  the  technical 
support  out  of  Deputy  for  Reconnaissance  and  Electronic  War¬ 
fare,  Aeronautical  Systems  Division,  ASD/RW,  which  is 
normally  collocated  from  ASD/RW.  ASD/RW  provides  avionics 
technical  support.  These  personnel  were  located  in  the  ATF 
SPO  and  reported  to  the  program  director. 

Interface  with  the  Prime  Contractors.  The  contractors 
participated  in  the  development  of  the  threat  description. 
They  were  active  members  of  the  various  working  groups. 

They  provided  the  required  data  and  performed  the  necessary 
trades.  They  provided  cost  data  for  the  Air  Force  budget 
models.  They  generated  the  appropriate  data  for  the  various 
reviews  which  was  used  "as-is”  in  briefings  to  the  highest 
levels  in  DOD  ( 1) . 

Integrated  Process  Team.  Although  IPDs  were  not 
formalized  until  FSED,  they  were  used,  informally,  by  the 
O&S  team  of  experts  to  involve  the  engineers  in  their 
meetings,  or  solving  their  problems  (33;  38). 

Acquisition  Process  Excellence  (APEX)  Study 

The  APEX  committee  was  established  by  General  Randolph 
as  a  follow-on  study  to  the  SPO  Sizing  Phase  I  initiative. 


44 


y 


There  were  four  objectives  of  the  initial  APEX  study. 

First,  the  study  attempted  to  baseline  the  entire 
acquisition  process  from  SON  to  Program  Management  Transfer 
Responsibility  (PMRT),  and  maintain/update  as  improvements/ 
streamlining  to  the  acquisition  process  were  identified 
(15).  Second,  it  analyzed  subprocesses  within  the 
acquisition  process  and  recommended  improvements  (15). 

Third,  the  study  identified  methods  to  streamline  the 
acquisition  process  (15).  Fourth,  it  identified  SPO  core 
tasks,  assigned  priorities,  and  defined  the  organization 
primarily  responsible  for  the  task  as  well  as  those  having 
collateral  responsibility  (15). 

The  study  was  completed  in  March  90.  A  major 
accomplishment  of  the  APEX  team  was  the  completion  of  the 
acquisition  flowchart  which  baselined  the  current 
acquisition  process.  Also,  more  than  320  improvement  ideas 
were  identified  from  interviews  with  program  directors, 
project  managers,  industry,  users,  functional  organization 
personnel ,  and  acquisition  executives  from  other  services 
(15).  As  these  improvement  ideas  were  identified,  the 
acquisition  flowchart  was  updated  by  ASD/XRM.  APEX 
developed  a  quality  function  deployment  action  plan  for  (1) 
developing/fielding  SPO  manpower  models;  and  (2) 
accomplishing  streamlining  within  a  defined  AFSC  Total 
Quality  Management  (TQM)  structure  (15).  The  APEX  team  also 
made  two  recommendations  as  a  result  of  their  initial  study. 
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First,  the  team  recommended  some  significant  changes  to  the 
way  a  SPO  manages  a  contract,  but  they  had  little  bearing  on 
the  topic  of  research  (15).  Briefly,  the  team  felt  that 
AFSC  spent  significant  resources  reviewing  and  approving 
lower-tiered  specifications  and  other  documents,  and 
intervening  in  solving  contractor  problems  (15). 

The  second  recommendation,  titled  Early  Acquisition 
Strategy  (EAS),  is  directly  related  to  the  topic  of 
research.  Based  on  the  APEX  team's  evaluation  of  the 
current  acquisition  process,  the  EAS  was  developed  as  an 
alternative  user-interface  strategy  to  address  the  study 
objective: 

The  current  process  of  developing  acquisition 
strategies  lacks  a  consistent,  systematic  approach,  is 
not  responsive  to  the  user,  does  not  comprehensively 
address  program  risks,  and  results  in  a  lack  of 
corporate  commitment  and  inexecutable  program.  (15) 

The  EAS  placed  the  user  in  charge  of  an  acquisition  team  in 

the  CE/D  phase  to  ensure  the  concepts  of  operation  are  fully 

developed  before  the  technical  solution  is  pursued  in  D/V 

and  FSD  (15).  Expanding  on  this  approach. 

The  concept  depends  heavily  upon  a  standard  Air  Force 
team  approach  to  conducting  acquisition  programs.  All 
Air  Force  MAJCOMS  must  have  a  common  understanding  so 
that  an  effective  team  is  formed  to  satisfy  the  using 
command's  need  for  an  operational  capability.  The  user 
should  lead  the  Air  Force  team  during  both  the  require¬ 
ments  definition  process  and  the  concept  exploration 
phase,  conducting  concept  direction  studies,  evaluating 
and  prioritizing  alternative  solutions,  and  advocating 
the  program  to  the  PDA.  This  could  be  accomplished  us¬ 
ing  an  organization  such  as  an  Acquisition  Support 
Group  (ASG) ,  chaired  by  the  user  and  established  prior 
to  milestone  0.  (15;  32;  39) 
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HQ  AFSC  XR  is  responsible  for  implementing  this  strategy, 
but  the  development  of  this  concept  is  still  in  the  early 
stages . 

The  APEX  accomplishments  along  with  the  implementation 
plan  for  the  EAS  and  contractor  accountability  recommenda¬ 
tions  provide  the  mechanisms  for  achieving  the  Air  Force 
goal  of  a  streamlined  acquisition  process  and  a  means  to 
define  manpower  requirements  and  allocate  resources  (17). 

APEX  II 

A  follow-on  streamlining  initiative  was  chartered  by 
General  Teal,  Vice  Commander  of  AFSC,  in  June  1990,  to 
demonstrate  the  feasibility  of  efficiently  identifying 
acquisition  process  opportunities.  The  following  redefined 
goals  were  developed  (15): 

1.  Baseline  the  existing  program  office  acquisition 
processes  and  update  as  the  process  is  improved. 

2.  Develop  metrics  that  allow  measurement  of  each 
process  and  knowledge  when  improvements  have  been  made. 

3.  Provide  each  field  commander  with  visibility  into 
those  processes  by  developing  "benchmarks"  for  those 
processes . 

4.  Share  with  all  of  AFSC  the  best  practices  found 
anywhere  in  the  acquisition  community. 

The  overall  results  of  achieving  these  goals  will  be  more 
and  smarter  streamlining  decisions  as  visibility  into  the 
acquisition  process  becomes  clearer  with  development  of  the 
acquisition  baseline,  useable  metrics,  and  benchmarks  (15). 
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Govermnent  Crew  System  Development  Study 

The  purpose  of  this  study  was  similar  to  the  first 
objective  of  the  APEX  study  which  was  to  develop  a  set  of 
core  tasks  for  acquiring  a  weapon  system  from  SON  submittal 
to  PMRT  (14).  The  responsible  government  office  is  the  CAT 
ADPO  which  is  located  at  WPAFB.  This  study  applied  to  new, 
major  weapon  system  programs.  This  study  was  useful  for  two 
reasons.  First,  the  contractor  performed  an  extensive 
literature  review  to  determine  what  Air  Force  documents  and 
policies  may  be  affected  by  changing  the  acquisition 
process.  Second,  the  contractor  used  the  ATF  program  to 
help  define  the  acquisition  players  and  core  tasks,  within  a 
program  office,  for  acquiring  a  weapon  system. 

Chapter  Summary 

The  purpose  of  Chapter  III  was  to  present  a  few  major 
topics  in  the  literature  related  to  the  acquisition  process. 
The  literature  covered  RSD  organizations  in  general  as  well 
as  a  review  of  three  related  studies  pertaining  to  improving 
the  acquisition  process  within  AFSC. 

Initially,  it  was  necessary  to  familiarize  the  reader 
with  the  objectives,  organizations,  and  key  players  and 
relationships  in  the  acquisition  ^tocBS3  and  TAF 
requirements  process.  The  TAF  requirements  process  is  a 
subprocess  of  the  acquisition  process. 
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In  section  II,  potential  improvement  activities 
associated  the  ATF  ESC  user-interface  process  were 
identified  and  discussed. 

Finally,  in  section  III,  Phase  I  and  Phase  II  of  the 
APEX  study  and  an  ASD  study  were  reviewed. 

The  initial  APEX  initiative  identified  numerous 
problems  with  the  way  AFSC  and  the  user  interface  in  the 
early  stages  of  the  acquisition  cycle,  as  viewed  by  program 
managers,  functional  experts,  users,  and  service  acquisition 
executives.  However,  their  Early  Acquisition  Strategy 
approach  is  really  limited  to  the  CE/D  phase. 

The  follow-on  APEX  initiative,  APEX  II,  concentrated  on 
demonstrating  the  feasibility  of  efficiently  identifying 
process  improvement  opportunities. 

The  CAT  study  really  did  not  apply  to  the  topic  of 
research.  Essentially,  a  local  contractor  has  developed  a 
model  for  defining  SPO  manpower  requirements  and  tasks  and 
allocating  SPO  resources. 

The  next  chapter  will  discuss  the  findings  of  this 
study . 
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Introduction 


IV.  Analysis  of  Data 


Since  the  objective  of  this  study  was  to  document  an 
existing  process,  the  ATF  ESC  user-interface  process,  in  a 
draft  handbook,  there  was  no  need  to  analyze  any  data. 

Senior  Air  Force  management,  recognizing  the  need  for  better 
communication  between  the  SPO  and  the  User,  directed  the  ATF 
SPO  to  develop  an  user-interface  process  which  would  improve 
the  effectiveness  of  the  D/V  phase  (1).  The  ATF  ESC 
interface  process  with  the  User  has  been  in  operation  for 
four  years,  and  it  has  been  accepted  by  all  of  the 
participating  organizations.  Upon  completion  of  the 
research  process,  however,  there  were  several  items  of 
interest  that  need  to  be  discussed. 

Acquisition  Process  Excellence  (APEX)  Study 

The  study  criteria  as  well  as  the  general  and  specific 
problems  were  defined  and  validated,  using  the  survey 
method,  by  the  APEX  evaluation  of  the  current  acquisition 
process.  The  criteria  for  improving  the  relationship 
between  the  SPO  and  the  User  were:  cooperation  and  exchange 
of  ideas,  disciplined  requirements  definition,  team 
approach,  exclusion  from  acquisition  process,  and  improved 
synergism  of  documentation  (16).  ATP  SPO  management  was 
aware  of  the  APEX  initiative,  but  the  concepts  associated 
with  the  ATF  ESC  interface  process  with  the  User  were 
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developed  before  it  began.  The  management  and  engineering 
techniques  associated  with  the  ATF  user-interface  approach 
did,  however,  fall  under  these  five  areas  of  concern. 

APEX  II 

The  APEX  committee  continues  to  work  with  functional 
experts  and  program  managers  throughout  the  Command  to 
identify  the  "best  practices"  in  any  acquisition  program  to 
enhance  the  acquisition  process.  This  follow-on  initiative 
was  called  APEX  II.  This  study,  or  a  follow-on  effort  from 
this  study,  could  be  pursued  through  the  APEX  II  initiative. 

Government  Crew  System  Development  Study 

The  objective  of  this  study  was  to  develop  a  SPO  team¬ 
building  model  for  organizing  and  identifying  specific 
responsibilities  and  functions  within  a  SPO  (14).  The  study 
really  produced  an  organizational  development  model.  It 
could  also  be  used  as  a  training  tool  for  new  SPO  members. 
Although  the  ATF  SPO  developed  some  unique  organizational 
techniques  associated  with  the  ATF  ESC  user-interface 
process,  the  model  was  not  used  in  this  study. 

F-15  Program 

Major  Gilbert  B.  Guarino  wrote  a  paper  on  lessons 
learned  from  the  P-15  program  (37:290).  In  fact,  the  ATF 
SPO  used  several  P-15  management  and  engineering  concepts, 
such  as  collocating  User-representatives  within  the  SPO, 
early  User  involvement,  and  system  definition  and  design,  in 
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its  user-interface  approach  (37:293).  However,  the  F-15 
approach  was  used  in  the  CE/D  phase  (37:293). 

ATF  ESC  Interface  Process  With  the  User:  Improvements 

Generally,  the  User  was  very  pleased  with  the  ATF 
ESC  user-interface  process.  There  were  a  couple  of  items 
that  the  User  felt  needed  improvement.  They  were  training 
and  the  number  of  User  personnel  in  SPO. 

Training .  Grade  restrictions  prevented  enlisted 
personnel  from  acquiring  the  necessary  training  for 
acquisition  related  work  (30;  31).  This  appears  to  be  an 
Air  Force  problem.  The  SPO  can,  however,  improve  the 
training  process  within  SPO  through  constant  communication, 
SPO  libraries,  and  acquiring  User  personnel  with  acquisition 
experience . 

Number  of  User  Personnel .  ATF  SPO  management  and  the 
Deputy  Program  Manager  for  Logistics  (DPML)  attempted  to 
acquire  an  O&S  expert  for  each  operational  Air  Force 
Specialty  Code  (AFSC)  (30;  31;  38).  If  successful,  they 
would  need  ten  OSS  experts.  Although  the  ATF  SPO  used  three 
OSS  experts,  in  addition  to  the  two  User-representatives, 
these  experts  were  still  overworked. 

Chapter  Summary 

The  User  was  extremely  satisfied  with  the  ATP  ESC 
interface  process.  The  User  had  two  concerns  with  the  ATP 
user-interface  approach  (training  and  the  number  of  User 
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personnel  assigned  to  the  SPO) ,  but  they  were  really  not  SPO 
problems.  In  addition,  the  ATF  ESC  user-interface  process 
was  developed  before  the  APEX  initiatives,  or  other 
acquisition  improvement  efforts  began.  Finally,  the  ATF  ESC 
user-interface  process  was  developed  primarily  from  lessons 
learned  on  past  programs,  and  specific  direction  from  senior 
Air  Force  leadership  (1;  35;  41;  46).  The  ATF  ESC  process 
was  successful  due  to  the  personalities  of  the  ATF  SPO 
leadership  and  personnel  (1). 
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V .  Conclusions  and  Recommendations 


Introduction 

The  purpose  of  this  chapter  was  to  discuss  specific 
conclusions  and  recommendations  from  the  study.  The 
conclusions  and  recommendations  were  not  very  general 
because  of  the  nature  of  the  study  and  its  limiting 
assitmptions .  The  conclusions  will  be  discussed,  first, 
followed  by  the  recommendations. 

Conclusions 

The  study  found  that  the  ATP  SPO  ESC  user-interface 
process  was  used  to  ensure  that  the  trade  study  process 
resulted  in  a  match  of  requirements  with  achievable 
technologies  to  baseline  an  executable  program  by  FSED  (1). 
The  process  was  pushed  by  senior  Air  Force  leadership.  Its 
concepts  were  based  on  lessons  learned  from  past  programs 
and  innovative  management  techniques. 

The  team-building  process  was  critical  in  the  D/V  phase 
because  participants,  such  as  the  O&S  personnel,  MPTS 
personnel ,  and  manufacturing  personnel  had  the  greatest 
opportunity  to  integrate  their  requirements  into  the  system 
design  ( 23) . 

In  addition,  the  ATF  ESC  process  concepts  could  be 
applied  to  every  major  program  in  the  D/V  phase.  The 
process  could  also  be  implemented  within  the  guidelines  of 
the  current  Air  Force  acquisition  process.  It  does  not 
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change  the  roles  and  responsibilities  of  the  SPO,  the  User, 
the  contractor,  or  the  other  participants.  It  was  also  used 
without  any  formal  agreement,  such  as  an  MOA,  between  the 
SPO  and  the  User. 

This  briefly  summarized  the  conclusions  from  the  study. 
The  following  paragraphs  cover  them  in  more  detail  by 
answering  the  investigative  questions. 

Investigative  Question  I.  What  factors  caused  the  ATF 
SPO  to  develop  the  ESC  interface  process  with  the  User?  The 
following  factors,  senior  Air  Force  management  guidelines, 
lessons  learned,  and  SPO  management  direction  contributed  to 
the  development  of  the  ESC  user-interface  process  (1).  Each 
factor  is  discussed,  separately,  next. 

Senior  Air  Force  Management  Guidance.  The 
commitment  by  senior  Air  Force  management  to  execute  the  ATF 
system  definition  and  design  concepts,  and  integrate  the 
User,  early,  in  the  acquisition  process  was  crucial  to  the 
success  of  the  ATF  ESC  user-interface  process  (1).  The  Air 
Force  Chief  of  Staff,  TAC  Commander,  AFSC  Commander,  ALD 
Commander,  and  SPO  director  were  all  committed  to  creating 
an  environment  of  openness  and  trust.  The  SPO  director  and 
his  User  counterparts  managed  to  create  an  environment  of 
openness  and  trust  through  many  innovative  management  and 
engineering  techniques  which  will  be  discussed  shortly. 

Lessons  Learned.  Based  on  lessons  learned  from 
other  programs,  it  is  easier  and  less  costly  to  integrate 
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manufacturing,  supportability ,  maintainability,  and  cost 
into  the  initial  design  while  it  is  evolving  (1).  This  can 
avoid  costly  weapon  system  modifications  and  upgrades  at  a 
later  date.  In  order  to  accomplish  this,  the  SPO  recruited 
program  managers,  engineers,  and  maintainers  with  recent 
experience  in  the  weapon  system  being  acquired.  They  were 
placed  in  key  management  and  engineering  positions.  User 
representatives  were  also  brought  into  the  SPO  at  the 
beginning  of  D/V.  In  addition,  the  maintenance  personnel 
were  given  the  latitude  (including  TDY  funding)  by 
management  to  acquire  the  necessary  support  from  any  expert 
within  DoD  and  the  defense  contractors  in  the  required 
discipline  areas,  such  as  ATF  Integrated  Maintenance  System 
(AIMS)  and  the  Integrated  Maintenance  Inspection  System 
(IMIS)  (30;  33;  38).  Also,  User  participation  in  the  PMRs , 
working  groups,  and  technical  meetings  was  part  of  the 
education  process,  and  was  crucial  to  a  program's  success. 

SPO  Management  Direction.  The  SPO  director  pushed 
the  use  of  state-of-the-art  technology  and  discriminate 
application  and  tailoring  of  military  standards  and 
specifications  (30).  For  example,  the  SORD  was  updated, 
annually,  rather  than  at  the  end  of  the  D/V  phase.  Test 
Information  Sheets  (TISs)  were  required  at  least  a  year  in 
advance  of  the  specific  test  date  instead  of  30-45  days  in 
advance.  This  ensured  that  the  test  objectives  were  clearly 
stated,  and  that  they  could  be  traced  back  to  a  specific 
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requirement  in  the  SORD.  Also,  the  SPO  director  desired  100 
percent  commonality  and  interoperability  between  the  ATF 
support  equipment  and  standard  Air  Forne  equipment  (35). 

Investigative  Question  II.  What  changes  were  required 
in  the  ATF  SPO's  acquisition  strategy  to  support  the  ATF  ESC 
interface  process  with  the  User?  Innovative  management 
concepts,  such  as  the  use  of  a  baseline  threat  description, 
a  baseline  TAC  BRAWLER  effectiveness  model,  a  relative  cost 
model,  Man-In-The-Loop  (MITL)  simulators.  Type  III  mockups, 
prototypes  and  discriminate  application  and  tailoring  of 
military  standards  and  specifications,  changed  the 
acquisition  strategy  (1;  20;  21;  30;  33;  38;  40).  Each  of 
these,  however,  were  discussed  in  more  detail  in  order  to 
answer  Question  3. 

Investigative  Question  III.  What  engineering  and 
management  techniques  were  developed  by  the  ATF  SPO  for  the 
ESC  interface  process  with  the  User?  The  techniques  will  be 
discussed  under  the  appropriate  concern  area  identified  by 
the  APEX  committee  survey.  They  were:  cooperation  and 
exchange  of  ideas,  disciplined  requirements  generation,  team 
approach,  exclusion  from  acquisition  process,  and  synergism 
of  documentation  (15). 

Cooperation  and  Exchange  of  Ideas.  The  following 
concepts  were  used  by  the  ATF  SPO  to  ensure  good  cooperation 
and  exchange  of  ideas  with  the  user. 
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Management  Commitment.  The  commitment  by 
senior  management  to  execute  the  ATF  system  definition  and 
design  concepts,  and  integrate  the  User,  early,  in  the 
acquisition  process  was  crucial  to  the  success  of  the  ATF 
ESC  user-interface  process  (!)•  The  Air  Force  Chief  of 
Staff,  TAC  Commander,  AFSC  Commander,  ALD  Commander,  and  SPO 
director  were  all  committed  to  creating  an  environment  of 
openness  and  trust.  The  SPO  director  and  his  User 
counterparts  also  managed  to  create  an  environment  of 
openness  and  trust  because  of  their  personalities  and 
innovative  management  techniques  (1). 

Early  User  Involvement.  Based  on  lessons 
learned  from  other  programs,  it  was  easier  and  less  costly 
to  integrate  manufacturing,  supportabi 1 ity ,  maintainability, 
and  cost  into  the  initial  design  while  it  was  evolving  (1). 
In  order  to  accomplish  this,  the  SPO  recruited  program 
managers,  engineers,  and  maintainers  with  recent  experience 
in  the  weapon  system  being  procured.  They  were  placed  in 
key  management  and  engineering  positions.  User 
representatives  were  brought  into  the  SPO  at  the  beginning 
of  D/V.  The  maintenance  personnel  were  given  the  latitude 
(including  TDY  funding)  by  management  to  acquire  and  use  any 
expert  within  DoD  and  the  defense  contractors  in  the 
required  discipline  areas,  such  as  AIMs  and  IMIS  (30;  33; 
38).  Also,  User  participation  in  PMRs ,  working  groups,  and 
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technical  meetings  was  part  of  the  education  process,  and 
was  crucial  to  a  program's  success. 

Organizational  Techniques.  The  ATF  SPO 
employed  some  unique  organizational  techniques  to  maintain 
control  of  the  program  and  the  funding,  and  to  remain  a 
credible  counterpart  to  the  user.  These  techniques  were: 

User  Representatives.  User 

representatives  from  ATC,  AFLC,  and  TAC  should  be  collocated 
in  the  ATF  SPO. 

O&S/MPTS  Experts.  The  DPML  attempted 
to  acquire  maintainability  experts  for  each  operational  AFSC 
(30;  33;  38;  41).  The  SPO  director  assembled  a  team  of 
training  experts  within  the  SPO.  The  team  consisted  of 
ASD/RW,  Training  System  Office  (TSO),  engineers,  and  ATC 
User-representatives . 

Experienced  SPO  Personnel .  Pi 1 ots , 

trainers,  and  maintainers,  with  current  and  extensive 
background  in  weapon  system  being  procured,  occupied  key 
engineering  and  management  positions  in  the  SPO. 

Engine  Responsibility.  The  engine 
program  is  generally  a  separate  program  from  the  weapon 
system.  However,  the  SPO  controlled  the  engine  program 
which  is  normally  managed  by  Deputy  for  Propulsion, 
Aeronautical  Systems  Division,  ASD/YZ  in  other  aircraft 
programs.  The  personnel  were  relocated  in  the  SPO  and 
reported  directly  to  the  Program  Director. 
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Avi oni cs  Support .  The  SPO  also 
controlled  the  technical  support  out  of  the  Deputy  for 
Reconnaissance  and  Electronic  Warfare,  Aeronautical  Systems 
Division,  ASD/RW.  ASD/RW  provides  avionics  technical 
support.  The  personnel  were  relocated  in  the  ATF  SPO  and 
reported  to  the  program  director. 

Laboratory  Technology.  The  SPO  signed 
Memorandums  of  Agreement  (MOA)  with  all  the  Air  Force 
laboratories.  The  ATF  SPO  established  and  controlled 
technology  panels  which  met  on  a  periodic  basis  and  involved 
AFSC,  AFSC  product  divisions.  Air  Force  laboratories.  User, 
ALDs,  and  flight  test  communities.  The  purpose  of  these 
panels  was  to  identify  any  technology  that  may  be  applicable 
to  the  system.  Many  of  these  technologies  and  technology 
programs  gave  the  SPO  and  the  User  better  insight  into 
potential  Pre-Planned  Product  Improvement  (PPPI)  technology 
candidates  for  the  ATF.  The  SPO  also  established  a 
technology  transition  working  group  with  the  contractor  to 
ensure  the  contractor  was  not  duplicating  the  efforts  of  the 
Air  Force  laboratories.  The  contractors  remained  informed 
about  technologies  that  could  be  used  on  the  weapon  system, 
but  they  were  not  allowed  to  use  the  lab  efforts  to  justify 
the  incorporation  of  technologies  into  the  baseline  through 
the  trade  study  process. 

Armament  Integration.  Munitions  System 
Division  (MSD)  established  an  operating  location  within  the 
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SPO  to  ensure  a  balanced  weapon  system  design.  A  Memorandum 
of  Agreement  (MOA)  was  written  to  specify  the  required 
support.  The  MSD  representative  provided  baseline  weapons 
data  to  ensure  total  weapon  system  integration,  and  ensured 
armament  program  cost,  schedule,  technical,  and  program 
direction  was  consistent  with  the  ATP  program.  The  SPO  also 
prepared  a  Weapons  Integration  Plan  which  guided  the 
activities  of  the  Armament  Integration  Working  Group  (ARWG) . 
The  ARWG  ensured  that  the  weapon  system  design  was  a 
weaponized  design. 

Training  Planning  Team.  SPOs  normally 
rely  on  MPTS  support  from  the  Training  Advisory  Office.  On 
the  other  hand,  the  ATP  SPO  established  a  team  of  MPTS 
experts  and  collocated  them  in  the  SPO.  The  team 
established  a  Training  Planning  Team  (TPT)  to  ensure  that 
MPTS  requirements  were  considered  in  the  design  process 
(41)  . 

Maintenance  Team.  The  ATP  SPO  used  the 
same  approach  for  the  O&S  requirements.  They  assembled  a 
team  of  experts  within  the  SPO.  This  team  also  developed  a 
Maintenance  Team  (MT)  to  ensure  that  the  requirements  were 
considered  in  the  design  (30;  33;  38). 

User  Personnel  Collocated  in  SPO.  Although 
the  primary  interface  was  through  TAC/DRB,  collocating  the 
User  personnel  in  the  SPO  was  helpful  to  the  SPO  when 
TAC/DRB  could  not  be  reached.  They  provided  continuity 
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between  the  User  and  their  SPO  counterparts.  They  opened  up 
the  lines  of  communications,  and  ensured  that  the  experts 
got  involved  (42). 

TACLO  User  Representative.  The  TACLO 
was  totally  integrated  into  the  SPO  activities.  He  attended 
staff  meetings,  PMRs,  technical  meetings,  and  briefings.  He 
participated  in  special  studies,  such  as  MAA/COEAs,  SABs , 
and  Multiple  Aircraft  Reviews  (MARs),  as  required. 

ATC  User  Representatives.  The  ATC  User- 
representative  was  totally  integrated  into  the  SPO 
activities.  He  attended  staff  meetings,  PMRs,  technical 
meetings,  and  briefings.  He  participated  in  the  TPT 
meetings . 

AFLC  User  Representatives.  The  DPML , 
with  support  from  the  SPO  director,  attempted  to  assign  a 
maintainability  specialist  to  each  operational  AFSC  (30;  33; 
38).  In  addition,  he  assigned  user  representatives  for 
each  AFSC,  or  at  least  in  the  AFSCs  that  he  is  unable  to 
fill  with  his  own  personnel  (30;  33;  38).  The  OSS  User- 
representatives  were  totally  integrated  into  the  SPO 
activities.  They  attended  staff  meetings,  PMRs,  technical 
meetings,  and  briefings.  They  participated  in  the  TPT 
meetings . 

Disciplined  Requirements  Generation. 

Disciplined  requirements  generation  means  integrating  cost, 
MPTS,  and  OSS  requirements  into  the  system  design  while  it 
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is  still  evolving.  This  required  early  integration  of  the 
User  into  the  acquisition  process,  and  educating  the  User 
(1;  34;  40).  It  resulted  in  a  match  of  requirements  and 
achievable  technologies  to  baseline  an  executable  program  at 
the  start  of  FSED  (1).  It  also  identified  PPPI 
technologies,  early,  so  that  the  weapon  system  was  designed 
to  defeat  an  evolving  enemy  (1).  Moreover,  it  allowed  the 
User  to  make  informed  decisions  (I). 

The  ATF  SPO  used  a  common  sense  approach  to  system 
design  and  definition  that  required  a  level  communication, 
openness,  trust,  and  interaction  between  the  engineers  and 
the  User  that  had  never  been  achieved  in  the  past.  Some  of 
the  techniques  used  by  the  ATF  SPO  were  as  follows: 
understand' the  TAC  requirements  and  use  the  best  design 
philosophy  to  achieve  them;  educate  and  involve  the  User 
through  the  trade  study  process.  Maintenance  Team  (MT) 
meetings,  and  Training  Planning  Team  (TPT)  meetings;  direct 
the  contractors  to  use  the  same  analytical  tools;  ensure  the 
Government  was  able  to  validate  the  Contractor's  results; 
ensure  the  Government  defined  the  mission  scenarios  and 
baselined  a  threat  description;  attempt  to  maintain 
commonality  with  existing  OSS  concepts;  and,  ensure  that  the 
Government  and  Contractor  organizations  developed  supporting 
documentation  (1;  20;  21;  30;  31;  33;  38;  40;  41). 

System  Definition  and  Design.  The  SPO 
focused  the  D/V  efforts  on  a  single  mission  requirement. 
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The  SPO  director  pushed  the  use  of  state-of-the-art 
technology,  and  discriminate  application  and  tailoring  of 
military  standards  and  specifications,  if  applicable  (30). 

He  also  insisted  on  100  percent  commonality  and 
interoperability  with  existing  O&S  and  MPTS  equipment  and 
concepts.  The  trade  study  process  required  extensive 
interaction  between  the  technical  people  and  the  User  in 
order  to  match  requirements  with  doable  technologies  (1). 

Trade  Study  Process.  The  following  process  was 
employed  very  successfully  by  the  ATP  SPO,  and  resulted  in  a 
match  of  requirements  and  achievable  technologies  to 
baseline  an  executable  program  by  FSED  (1).  It  also 
identified  PPPI  concepts  that  were  accounted  for  in  the 
design.  The  process  was  as  follows  (1;  34;  40): 

1.  It  started  out  with  a  number  of  generic  concepts,  based 
on  the  assumption  that  certain  technologies  would  be 
available  at  the  appropriate  time,  and  broadly  stated  User 
requirements . 

2.  Then,  the  User  choose  a  specific  design  concept. 

3.  And  this  was  where  the  education  process  began: 

a.  Through  interaction  between  the  engineers  and  the 
User,  the  User  was  educated  on  what  was  doable. 

b.  Through  analysis  and  testing,  the  engineers  came  up 
with  design  concepts,  based  on  the  broadly  stated  User 
requirements,  and  the  associated  cost,  technical, 
manufacturing,  MPTS,  and  O&S  characteristics.  The  cost  and 
technical  data  was  derived  from  a  relative  sensitivity  cost 
model  and  the  TAC  BRAWLER  effectiveness  model . 

c.  Based  on  this,  the  User  decided  on  the  performance 
tradeoffs . 
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d.  The  above  process  was  repeated  with  the  SPO 
providing  design  alternatives  with  supporting  data  and  the 
user  making  the  performance  tradeoffs. 

4.  The  SPO  also  identified  dates  when  the  avionics, 
aircraft,  and  engine  designs  needed  to  be  finalized. 

5.  The  end  result  was  a  data  base  that  matched  requirements 
with  doable  technologies  (with  growth  designed  in  for  PPPI 
options)  to  baseline  an  executable  program  at  PSED  (1). 

Integrated  Design  Approach.  By  integrating 
the  trade  study  activities,  with  the  test,  O&S,  and  MPTS 
activities,  the  final  design  was  not  only  achievable,  but  it 
was  an  integrated  design.  The  ATP  used  the  following 
concepts  to  achieve  this  level  of  design  risk  and 
integration. 

Test  Philosophy.  Analysis  and  test 
played  a  crucial  role  in  verifying  and  documenting  the 
doable  technologies  as  well  as  the  potential  PPPI 
technologies.  The  following  test  concepts  were  employed  by 
the  ATP  SPO  in  the  D/V  phase  of  the  program: 

Team  Concept.  The  test  manager 
ensured  that  the  User,  Contractor,  O&S,  MPTS,  APOTEC,  flight 
test  center,  and  ALD  communities  were  involved  in  PMR, 
working  group,  test  planning,  and  technical  meetings. 

Test  and  Evaluation  Master  Plan. 

The  test  manager  ensured  that  the  User,  Contractor,  O&S, 
MPTS,  APOTEC,  flight  test  center,  and  ALD  communities  were 
involved  in  the  preparation  and  review  of  the  Test  and 
Evaluation  Master  Plan  (TEMP). 
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Class  III  Mockups.  Since  the 

prototypes  are,  typically,  not  available  until  late  in  the 
D/V  program,  the  contractor  built  Class  III  mockups  for 
supportability  and  maintainability  demonstrations  and  MPTS 
efforts  (38). 

"Ply-Before-Buy"/Prototypes .  The 
SPO  used  a  "fly-bef ore-buy”  approach  by  testing  with 
prototypes  before  committing  funds  to  FSED. 

Avionics  Flying  Lab.  The 

contractors  used  an  Avionics  Flying  Lab  (AFL)  to  test 
avionics  design  concepts  not  planned  to  be  tested  on  the 
prototypes.  The  contractor  also  performed  full-scale  ground 
tests . 

Ground  Testing.  The  contractor 
used  ground  testing  to  test  systems,  subsystems,  aiid 
components  that  were  not  tested  on  the  prototype.  Ground 
testing  also  supported  flight  testing,  especially  for  the 
high-risk  tests. 

Test  Information  Sheets.  The  SPO 
required  draft  Test  Information  Sheets  (TISs)  on  all  tests 
to  be  delivered  at  least  12  months  in  advance  of  flight 
testing . 

Engineering  Review  Teams.  The  SPO 
encouraged  and  supported  independent  Engineering  Review 
Teams  (EITs)  for  all  test  activities.  The  primary  concern 
of  these  teams  was  to  ensure  that  the  testing  supported 
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higher- level  SORD  requirements,  and  that  it  resulted  in 
adequate  risk  reduction.  SPO  and  User  representatives 
attended  these  reviews  as  well. 

Training  Planning  Team  (TPT).  The  ATF 
SPO  used  a  team  of  training  experts  within  the  SPO  to  ensure 
MPTS  requirements  were  integrated  into  the  design  (41). 

They  established  a  TPT  to  acquire  inputs  from  experts 
throughout  OoD  and  the  defense  contractors  (41). 

Maintenance  Team  (MT).  The  ATP  SPO  used 
a  team  of  maintenance  experts  within  the  SPO  to  ensure  O&S 
requirements  were  integrated  into  the  design  (30;  33;  38). 
They  established  a  MT  to  acquire  inputs  from  experts 
throughout  DoD  and  the  defense  contractors  (30;  33;  38). 

Budget  Model .  A  relational,  detailed  budget 
model  was  developed  to  validate  the  contractor  data  in 
relative  terms,  and  cost  proposed  configurations  as  they 
were  developed  to  match  the  latest  set  of  requirements.  The 
model  was  limited  to  just  Air  Force  data.  Contractor  data 
was  also  used  (23;  35). 

MOAs  and  Working  Group  Charters.  There  were 
a  large  number  of  organizations  supporting  the  program 
office.  The  program  director  ensured  that  these  documents, 
if  used,  did  not  restrict  or  limit  the  technical  support 
from  the  participating  agencies. 

Technology  Transition.  In  order  to  minimize 
cost  and  properly  integrate  OSS  and  MPTS  requirements  into 
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the  design,  the  SPO  identified  potential  PPPI  improvements 
as  early  as  possible.  These  potential  improvements  were 
docTimented  and  tracked  by  a  SPO  manager  using  fact  sheets. 
The  SPO  also  established  a  Technology  Panel  and  a  Technology 
Transition  Working  Group  to  keep  the  Government  and  the 
Contractors  informed  about  potential  PPPI  technologies. 

Contractor  Interface.  The  contractor  was 
responsible  for  performing  the  trade  studies.  This  process 
included  acquiring  sufficient  data  to  make  requirements 
decisions  and  maturing  technologies  with  which  to  enter  PSD, 
both  risk  reduction  activities  (13).  This  was  an  iterative 
process  in  which  the  SPO  provided  the  contractors  a  system 
requirement  against  which  the  contractors  applied  trade 
studies  and  demonstration  tools  (13).  The  contractors  then 
returned  their  recommendations  for  changes  to  the  require¬ 
ment  for  better  system  optimization  (13).  These  require¬ 
ments  recommendations  were  analyzed  by  the  Air  Force. 
Accepted  recommendations  were  documented  in  the  system 
specification  and  returned  to  the  contractor  as  new 
requirements.  This  cycle  was  repeated  annually  during  the 
D/V  phase.  The  contractor  also  participated  in  the  TPTs , 

MTs ,  working  groups,  PMRs ,  and  special  studies.  The 
contractor  was  an  accountable  team  member.  The  contractor 
was  also  required  to  establish  subcontracts  and  Associate 
Contractor  Agreements  (ACAs)  with  other  weapon  system 
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contractors,  and  develop  draft  Interface  Control  Documents 
(ICDs)  for  FSED. 

Team  Approach.  In  the  case  of  the  ATP  program,  a 
successful  team  acquisition  strategy  resulted  from  good 
communication  and  exchange  of  information  and  a  disciplined 
requirements  generation  approach.  The  ATP  used  innovative 
management  and  organizational  concepts  in  these  areas  as 
well  as  a  team-acquisition  strategy  through  working  groups, 
PMRs,  technical  meetings,  SPO  maintenance  teams,  and  SPO 
training  teams.  This  was  also  accomplished  without  MOAs . 

Exclusion  from  Acquisition  Process.  Two  aspects 
of  the  ATP  user-interface  philosophy  precluded  the  SPO  from 
excluding  the  User  from  the  acquisition  process.  First,  SPO 
management  believed  that  if  the  requirement  was  not  in  the 
SORD,  then  it  was  not  a  requirement  (1).  Second,  the  SPO 
believed  that  the  trade  study  process  was  a  education 
process  which  would  help  the  User  make  informed  decisions 
regarding  the  cost,  schedule,  and  performance  tradeoffs  (1). 
The  User  was  involved  in  staff  meetings,  program  reviews, 
special  studies,  and  technical  meetings.  On  the  other  hand, 
the  User-representatives  felt  they  lacked  the  appropriate 
training  before  they  entered  the  program  office. 

Synergism  of  Documentation.  The  following 
documents  and  analytical  tools  formed  the  database  that  was 
used  by  the  SPO  to  answer  inquiries,  and  to  justify  the 
requirements,  the  trade  study  results,  and  the  PPPI 
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technologies.  They  also  ensured  that  there  was  traceability 
throughout  the  program  documentation  as  the  requirements 
were  being  refined.  Most  importantly,  they  constituted  the 
data  base  used  to  prove  that  there  is  a  match  between  the 
requirements  and  the  associated  technology. 

In  order  to  accomplish  these  things,  the  SPO  developed 
a  matrix  which  cross  referenced  each  requirement  through  all 
of  these  documents  (38).  In  addition,  the  SPO  established  a 
group  of  people  to  periodically  update  the  matrix,  and 
ensure  that  the  Preliminary  System  Specification  (PSS)  and 
the  System  Operational  Requirements  Document  (SORD)  matched 
one-for-one  (38).  The  team  included  SPO,  User 
representative,  engineering,  TAC/DRB,  AFOTEC,  HQ  AFSC, 
SAF/AQL/AQR,  ATC,  and  AFLC  personnel. 

SPO  Library.  The  SPO  was  developing  a  SPO 
library.  This  library  documented  lessons  learned,  and  was 
used  to  train  SPO  personnel  and  User-representatives,  and  to 
maintain  continuity. 

Program  Management  Direction  (PMD).  The 
User,  HQ  AFSC,  and  SAF/AQ  representatives  attended  working 
groups,  PMRs,  and  technical  meetings.  This  enabled  the  SPO 
to  get  the  proper  direction,  if  required.  The  headquarters 
personnel  communicated  with  the  SPO  on  a  daily  basis,  and 
came  TDY  at  least  once  a  month. 
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Preliminary  System  Specification  (PSS).  The 
SPO  and  the  User  ensured  that  the  Draft  PSS  and  the  SORD 
matched  one-£or-one  before  the  PSS  was  sent  to  the 
contractors.  The  SPO  and  the  User  periodically  reviewed  the 
PSS  and  SORD  to  ensure  that  they  still  matched.  The  PSS  was 
also  updated  when  the  User  made  performance  tradeoffs  as 
described  in  the  trade  study  process  (23).  In  addition,  the 
working  groups,  technical  meetings,  TPT  meetings,  and  the  MT 
meetings  generated  and  documented  proposed  changes.  These 
changes  were  eventually  incorporated  into  PSS  and  SORD. 

System  Operational  Requirements  Document. 

The  System  Operational  Requirements  Document  (SORD)  was 
updated  after  each  tradeoff  decision  point  rather  than  at 
the  end  of  D/V,  or  a  milestone  decision  point  (23). 

Trade  Study  Results.  The  contractor 
referenced  the  PSS  and  SORD  paragraphs  for  each  requirement, 
or  trade  study  initiative.  The  contractor  developed  control 
charts  for  each  SORD  and  PSS  requirement.  These  charts  also 
delineated  the  risk  reduction  activities  (analysis,  test,..) 
required  to  reduce  the  risk  of  the  technology  to  an 
acceptable  level  so  that  they  could  be  incorporated  into  the 
baseline.  Also,  the  Contractor  discussed  any  requirements 
that  were  not  achievable  with  current  or  near-term 
technol ogi -is  in  the  PMRs  and  technical  meetings. 

Special  Studies.  Special  studies,  such  as 
Science  Advisory  Boards  (SABs),  Mission  Area  Analysis  and 
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Operational  Cost  and  Effectiveness  Analysis  (MAA/COEA) , 

AFSC,  or  User-directed  studies,  occurred  in  the  ATF  D/V 
phase  (23).  The  above  documentation  reflected  the  results 
from  these  studies,  if  required.  In  addition,  existing 
tools  and  the  database  were  used  to  accomplish  these 
studies . 

Science  Advisory  Board.  The  SPO  did  not 
need  to  generate  any  additional  data,  or  develop  new 
analytical  tools  for  this  effort.  The  SPO  integrated  the 
User,  AFOTEC,  Foreign  Technology  Division  (FTD) ,  Air  Force 
Chief  of  Staff /Studies  and  Analysis  (AFCS/S&A) ,  Defense 
Intelligence  Agency  (DIA) ,  and  AF/IN  into  this  effort  to, 
primarily,  coordinate  the  information. 

MAA/COEAs .  The  SPO  did  not  need  to 
generate  any  additional  data,  or  develop  new  analytical 
tools  for  this  effort.  The  SPO  integrated  the  User,  AFOTEC, 
FTD-,  AFCS/S&A,  DIA,  and  AF/IN,  into  this  effort  to, 
primarily,  coordinate  the  information. 

Inquiries  from  External  Sources.  The 
ATF  SPO  was  able  to  address  these  inquiries  very  quickly 
with  the  matrix  and  the  trade  study  database. 

Simulation  Data.  Simulation  is  an  analytical 
tool  which  can  support  a  number  of  efforts  that  could  arise 
during  the  D/V  phase,  such  as  MAA/COEAs,  sensitivity 
analyses,  SABs ,  and  special  studies,  directed  by  outside 
agencies.  More  importantly,  they  were  used  by  the  ATF  SPO 
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to  validate  the  contractor  effectiveness  analysis  results, 
and  to  measure  the  tactical  utility  of  a  given  weapons 
system,  or  subsystem  design  (1).  The  simulation  tools,  such 
as  TAG  BRAWLER,  incorporated  the  most  current  baseline 
threat,  mission  scenario,  and  Man-In-The-Loop  (MITL)  data. 
Successful  baselining  and  integration  of  these  activities 
added  justification  and  traceability  to  the  requirements 
generation  process.  This  area  was  always  a  topic  of  the 
quarterly  program  reviews. 

Threat .  The  program  director  ensured 
that  the  Government  and  the  Contractor  were  using  a 
baselined  threat  description.  He  also  ensured  that  the  User 
and  the  appropriate  participating  agencies  were  involved  in 
baselining  the  threat  description.  In  order  to  accomplish 
this,  the  SPO  established  a  Threat  Working  Group  (TWG)  to 
control  and  coordinate  this  activity. 

Mission  Scenarios.  The  program  director 
ensured  that  the  Government  and  the  Contractor  were  using 
approved  mission  scenarios  in  the  D/V  activities.  He 
ensured  that  the  User  and  the  appropriate  participating 
agencies  were  involved  in  definition  and  approval  of  the 
mission  scenarios.  In  order  to  accomplish  and  control  this 
process,  the  SPO  established  a  Mission  Scenario  Group  (MSG). 

MAN-IN-THE-LOOP  (MITL).  The  SPO 
established  and  coordinated  a  MITL  program.  The  purpose  of 
the  MITL  activity  was  to  obtain  expert  feedback  from  pilots. 
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operational  and  test,  on  the  scenarios,  tactics,  and  the 
cockpit  development  effort  (42). 

TAG  BRAWLER.  The  program  director 
ensured  that  the  Government  and  the  Contractor  were  using 
the  same  version  of  TAG  BRAWLER.  He  ensured  that  the  user 
and  the  participating  organizations  were  involved  in  the 
definition  and  the  baselining  of  TAG  BRAWLER  model  for  the 
D/V  effort.  In  order  to  accomplish  this,  the  SPO 
established  a  TAG  BRAWLER  Group  (TBG)  to  control  this 
effort . 

Other  Simulations.  The  SPO  also 
contracted  for  independent  analyses  of  the  prime 
contractor's  effectiveness  study  results  (Mission 
Effectiveness  Analysis),  and  for  separate  in-house  studies. 
The  SPO  and  the  User  approved  the  analytical  tools  that  were 
used  by  other  contractors. 

Investigative  Question  IV.  How  can  these  engineering 
and  management  techniques  be  generalized  and  applied  to 
other  major  Air  Force  acquisition  programs? 

Handbook .  The  most  appropriate  method  would  be 
through  a  handbook.  The  objective  of  this  study  was  to 
develop  such  a  draft  handbook.  There  was  not  enough  time  in 
this  study  to  coordinate  the  handbook  with  the  appropriate 
organizations,  including  the  ATF  SPO,  and  to  get  it  into  the 
proper  format.  In  addition,  additional  study  would  be 
required  to  expand  the  handbook  into  a  training  tool. 


Journal  Article.  Again,  this  is  outside  the  scope 
of  this  study,  but  an  article  could  be  written  with  the 
assistance  of  the  thesis  advisor,  and  approval  by  the  ATF 
SPO. 

Briefing .  The  results  of  the  study  could  be 
sununarized  into  a  briefing  for  the  APEX  committee  and  other 
ASD  program  directors  but  the  idea  would  have  to  be 
coordinated  with  the  ATF  SPO. 

Investigative  Question  V.  Into  which  acquisition  phase 
should  the  program  director  implement  the  ATF  interface 
process  with  the  User?  (The  production  and  deployment  phases 
do  not  apply  to  this  question.) 

Concept  Exploration  and  Definition  Phase.  Many  of 
the  concepts  and  techniques  do  not  apply  to  the  CE/D  phase 
due  the  limiting  assumptions.  A  SPO  is  not  formed  until 
D/V.  Also,  the  functional  participants  that  participated  in 
the  ATF  ESC  approach  are  not  in  place  in  the  CE/D  phase. 
Finally,  the  level  system  definition  and  design  and  threat 
information  that  would  be  needed  to  design  or  baseline  the 
database  and  the  analytical  tools  would  not  be  available  in 
this  phase. 

Demonstration  and  Validation  Phase.  The  handbook 
was  designed  to  assist  a  program  director  with  a  program  in 
the  D/V  phase.  Please  refer  to  the  research  objective. 
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Full  Scale  Engineering  and  Development  Phase.  A 


separate  study  would  be  required  to  determine  the 
applicability  of  the  ATF  ESC  user-interface  process  concepts 
and  techniques  to  FSED.  The  ATP  SPO  planned  to  use  many  of 
the  same  techniques  in  FSED. 

Investigative  Question  VI.  What  approval  cycle  is 
required  to  implement  the  ATF  ESC  interface  process  with  the 
User?  The  ATF  ESC  user-interface  process  concepts  and 
techniques  were  developed,  in  part,  from  Senior  Air  Force 
leadership  direction.  The  process  has  been  in  operation  for 
four  years,  and  it  has  been  accepted  by  the  implementing, 
supporting,  participating,  and  supporting  command 
representatives  involved  in  the  ATF  program.  It  was  also 
assumed,  up  front,  that  the  ATF  ESC  interface  process  could 
be  used  by  other  major  acquisition  programs  in  the  D/V 
phase.  Therefore,  a  comprehensive  review  of  Air  Force 
policy  and  guidance  was  outside  the  scope  of  this  study. 

Recommendations 

The  study  recommended  that  the  ATF  ESC  user-interface 
process  handbook  be  formalized  and  provided  to  other  SPOs  as 
a  management  tool  for  establishing  an  effective  SPO-User 
relationship.  At  a  minimum,  this  will  require  additional 
research,  and  a  thorough  review  by  the  ATF  SPO.  Additional 
interviews  will  also  be  required  to  understand  the  proper 
format  for  the  handbook,  and  the  process  of  formalizing  the 
handbook  through  Air  Force  channels.  Meanwhile,  a  technical 
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paper  could  be  published  in  Air  Force  journals.  Or, 
briefings  could  be  given  to  APEX  committee  participants,  or 
SPO  directors  with  new  programs  at  ASD.  Currently,  however, 
it  is  assumed  that  the  draft  handbook  applies  to  major 
programs  in  the  D/V  phase. 

Recommendations  for  Further  Research 

The  study  also  proposed  some  additional  research  that 
could  be  related  to  the  APEX  initiatives  and  this  study. 

Each  of  these  topics  will  be  discussed,  separately,  next. 

Training  Handbook.  The  User  expressed  an  interest  in 
using  the  ATP  ESC  interface  process  with  the  User  handbook 
as  a  training  tool,  much  like  the  TAP  Requirements  Process 
Handbook  (52).  This  would  require  additional  information  on 
the  acquisition  process  to  be  incorporated  into  the  draft 
ATF  ESC  user-interf ace  process  handbook. 

AFSC/AFLC  Merger.  The  ATF  ESC  user-interface  process 
concepts  would  still  apply.  However,  additional  research 
would  be  required  to  address  the  applicability  of  some  the 
organizational  methods  used  by  the  ATF  SPO. 

General  Applicability  of  Handbook.  The  study  was  based 
on  some  limiting  assumptions  that  have  already  been 
discussed.  Currently,  it  is  applicable  to  the  D/V  phase  of 
major  acquisition  programs.  The  acquisition  process  was 
also  presented  from  an  ATF  SPO  point-of-view.  Therefore,  a 
comprehensive  review  of  Air  Force  acquisition  policy  and 
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documentation  to  determine  the  appropriateness  of  the  user- 
interface  concepts  would  need  to  be  done.  In  addition,  the 
handbook  may  already  need  to  be  updated  due  the  changing 
nature  of  Air  Force  acquisition  guidelines  and  terminology. 

Applicability  of  Handbook  to  FSED.  The  ATF  SPO  planned 
to  use  the  same  approach  in  FSED.  However,  additional 
research  would  be  required  to  determine  if  any  changes  would 
need  to  be  made  to  the  current  draft  handbook. 

Effectiveness  Of  ATF  ESC  User- Interface  Process.  After 
the  ATF  program  is  two  to  three  years  into  FSED,  the 
effectiveness  of  the  ATF  ESC  process  could  be  measured 
against  the  F-15  and  F-16  programs  by  comparing  the  type  and 
number  of  changes  to  the  technical /operational 
characteristics  documented  in  the  Selected  Acquisition 
Reports  (SARs). 

Chapter  Summary 

The  study  recommended  that  the  handbook  be  formalized 
and  provided  to  other  SPOs  as  a  management  tool  for 
establishing  an  effective  SPO-User  relationship.  The 
handbook  could  be  modified  and  used  as  a  SPO  training  tool 
with  additional  research  and  documentation.  The  handbook 
could  also  be  modified  to  address  the  AFSC/AFLC  merger  as 
well  as  FSED,  but  additional  research  would  need  to  be 
accomplished.  Finally,  after  the  ATF  program  is  two  to 
three  years  into  FSED,  the  effectiveness  of  the  ATF  ESC 
process  could  be  measured  against  the  F-15  and  F-16  programs 
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by  comparing  the  nvunber  and  type  of  changes  to  the 
technical/operational  characteristics  documented  in  the 
Selected  Acquisition  Report  (SAR). 
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APPENDIX  A 


ADVANCED  TACTICAL  FIGHTER  (ATF) 
EARLY,  STRUCTURED,  AND  CONTINUOUS  (ESC) 
SPO-USER  TEAM-BUILDING  PROCESS 
DRAFT  HANDBOOK 


This  handbook  is  divided  into  eleven  sections.  The 
introduction  is  in  Section  1.0.  A  complete  listing  of  the 
study  assumptions  is  in  Section  2.0.  The  study  criteria, 
which  were  identified  by  the  Acquisition  Process  Excellence 
(APEX)  Team,  are  discussed  in  Section  3.0.  A  summary  of  the 
ATF  program  is  in  Section  4.0.  The  ATF  ESC  user-interface 
process  is  briefly  discussed  in  Section  5.0.  The  reference 
documents  are  listed  in  Section  6.0.  The  ATF  management  and 
engineering  concepts  that  respond  to  the  first  APEX 
committee  finding,  good  cooperation  and  exchange  of  ideas, 
are  identified  and  discussed  in  Section  7.0.  The  ATF 
management  and  engineering  concepts  that  respond  to  the 
second  APEX  committee  finding,  disciplined  requirements 
generation,  are  identified  and  discussed  in  Section  8.0. 

The  ATF  management  and  engineering  concepts  that  respond  to 
the  third  APEX  committee  finding,  team  approach,  are 
identified  and  discussed  in  Section  9.0.  The  ATF  management 
and  engineering  concepts  that  respond  to  the  fourth  APEX 
committee  finding,  exclusion  from  acquisition  process,  are 
identified  and  discussed  in  Section  10.0.  Finally,  the  ATF 
management  and  engineering  concepts  that  respond  to  the 
fifth  APEX  committee  finding,  synergism  of  documentation, 
are  discussed  in  Section  11.0. 
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LIST  OF  ACRONYMS /DEFINITIONS 


In  order  to  understand  any  aspect  of  the  acquisition 
process,  which  includes  the  operational  requirements 
process,  it  is  necessary  to  familiarize  the  User  with  some 
related  terminology.  These  are: 

1.  Acquisition  process.  A  term  used  to  cover  the  entire 
process  of  acquiring  weapons  from  requirement  generation, 
research,  development,  test  and  evaluation,  production, 
operational  support,  product  improvement,  and  final 
disposal . 

2.  Air  Force  Flight  Test  Center  (AFFTC) .  Performs 
experimental  flight  testing  of  aircraft  and  aerospace 
vehicles  entering  the  Air  Force  inventory  or  that  of  other 
services . 

3.  Air  Force  Logistics  Command  (AFLC).  Major  Air  Force 
command  responsible  for  logistical ly  supporting  all  Air 
Force  activities  as  well  as  other  DoD  and  federal  agencies. 
The  supporting  command  (19:68).  Also  known  as  the 
supporting  command. 

4.  Air  Force  Operational  Test  and  Evaluation  Command 
(AFOTEC) .  Responsible  as  the  Air  Force's  independent 
manager  for  operational  test  and  evaluation  (OT&E)  of  weapon 
systems  and  other  hardware  (19:112) 

5.  Air  Force  Systems  Command  (AFSC) .  Major  Air  Force 
command  responsible  for  total  Air  Force  research, 
development,  testing,  evaluation,  and  contracting  for 
production  of  aerospace  systems.  Also  known  as  the 
implementing  command  (14:Vol  VI I, 21). 

6.  Air  Training  Command  (ATC) .  Major  air  command 
responsible  for  training  aircrews  and  maintenance  personnel. 
Also  known  as  the  participating  command. 

7.  Aeronautical  Systems  Division  (ASD).  Division  of  Air 
Force  Systems  command  responsible  for  directing  the  design, 
development,  and  acquisition  of  major  aerospace  systems. 

8.  Armament  Working  Group  (ARWG) .  Consists  of  project 
managers,  engineers,  pilots.  Operation  and  Support  (O&S) , 
Manpower  Personnel  and  Training  System  (MPTS),  and 
contractor  personnel  who  are  responsible  for  providing 
recommendations  to  the  program  director  on  armament 
integration  issues  (3;  4;  5;  11;  12). 

9.  Avionics  Working  Group  (AWG).  Consists  of  project 
managers,  engineers,  pilots.  Operation  and  Support  (O&S), 
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Manpower  Personnel  and  Training  System  (MPTS),  and 
contractor  personnel  who  are  responsible  for  providing 
recommendations  to  the  program  director  on  avionics  issues 
(3;  4;  5;  11;  12). 

10.  ASD/YF.  The  Advanced  Tactical  Fighter  System  Program 
Office.  Organization  within  ASD  responsible  for  managing 
the  design,  development,  and  acquisition  of  the  Advanced 
Tactical  Fighter  (ATF) . 

11.  ATF  Integrated  Maintenance  System  (AIMS). 

12.  Air  Logistics  Center  (ALC) .  Organization  within  AFLC 
assigned  system  and  commodity  responsibilities  on  a 
worldwide  support  basis  for  specified  systems  (19:80). 

13.  Associate  Contractor  Agreement  (ACA) .  An  agreement 
between  two  or  more  contractors  (government  not  involved)  to 
provide  top-level  cost,  schedule,  and  performance  data, 
answer  phone  calls,  and  attend  meetings. 

14.  Computer-Aided  Acquisition  and  Logistics  (CALS). 

15.  Cockpit  Working  Group  (CWG) .  Consists  of  project 
managers,  engineers,  pilots.  Operation  and  Support  (O&S), 
Manpower,  Personnel,  Training,  and  Safety  (MPTS),  and 
contractor  personnel  who  are  responsible  for  providing 
recommendations  to  the  program  director  on  cockpit  issues 
(3;  4;  5;  11;  12). 

16.  Concept  Exploration/Definition  (CE/D)  phase.  Weapon 
system  acquisition  phase  where  a  SPO  explores  generic  design 
alternatives  to  satisfy  a  stated  mission  need. 

17.  Cockpit  Automation  Technology  (CAT).  Advanced 
development  program  responsible  for  development  of  a  highly 
disciplined  and  structured  crew  system  design  process 
including  the  supporting  design  tools  and  technology  (14:Vol 
VII, 6) . 

18.  Cost  Model.  Estimating  tool  consisting  of  one  or  more 
cost  estimating  relationships,  estimating  methodologies,  or 
estimating  techniques  used  to  predict  the  cost  of  a  weapon 
system  and  its  components  (14:Vol  VII, 14). 

19.  Defense  Acquisition  Board  (DAB).  Top  level  DoD 
corporate  body  for  system  acquisition  responsible  for 
providing  advice  and  assistance  to  the  Secretary  of  Defense. 
Reviews  the  weapon  system  acquisition  at  milestone  decision 
points  (14:Vol  VII, 22). 
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20.  Demonstration/Validation  (D/V)  phase.  Weapon  system 
phase  where  program  characteristics  (performance,  cost,  and 
schedule)  are  validated  and  refined  through  extensive  study 
and  analysis,  hardware  development  and  prototype  testing. 
{14:Vol  VII ,22) . 

21.  Defense  Management  Review  (DMR). 

22.  Developmental  Test  and  Evaluation  (DT&E).  Verifies  the 
technical  performance  specifications  and  objectives  are  met. 
Includes  test  and  evaluation  (T&E)  of  components, 
subsystems,  hardware/software  integration,  related  software, 
and  models  of  the  system.  T&E  of  compatibility  and 
interoperability  with  existing  and  planned  equipment  and 
systems  is  also  included  (14:Vol  VII, 26). 

23.  Design  to  Cost  (DTC).  A  management  concept  wherein 
rigorous  cost  goals  are  established  during  development  and 
the  control  of  system  costs  (acquisition,  operating,  and 
support)  to  these  goals  is  achieved  by  practical  tradeoffs 
between  operational  capability,  performance,  cost,  and 
schedule.  Cost,  as  a  key  design  parameter,  is  addressed  on 
a  continuing  basis  and  as  an  inherent  part  of  the 
development  and  production  process  (14:Vol  VII, 26). 

24.  Depot  Training  System  (DTS). 

25.  Engineering  Change  Proposal  (ECP) .  Formal  proposal  by 
contractor  for  modification  to  the  weapon  system. 

26.  Full  Scale  Engineering  Development  (FSED).  The  weapon 
system  acquisition  phase  where  the  weapon  system  is 
designed,  fabricated,  and  tested. 

27.  Foreign  Technology  Division  (FTD) .  Responsible  for 
acquiring,  collecting,  analyzing,  producing,  and 
disseminating  foreign  aerospace  scientific  and  technical 
intelligence  related  to  current  capabilities  and  potential 
threats  of  major  adversary  powers  in  support  of  AFSC  and 
other  agency  requirements  (19:64). 

28.  Government  Furnished  Equipment  (GFE) .  Items  in 
possession  of,  or  acquired  by,  the  Government  and 
deliverable  to  or  made  available  to  the  contractor  (14:Vol 
VII, 39). 

29.  Implementing  Command.  See  Air  Force  Systems  Command. 

30.  Interface  Control  Document  (ICD).  Contractor-generated 
documents  which  describe  the  means  (hardware,  software,  and 
data)  by  which  the  weapon  system  components  are  interfaced 
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with  each  other,  with  other  subsystems,  and  with  planned 
systems . 

31.  Initial  Operational  Test  and  Evaluation  (lOT&E). 

Testing  which  provides  estimate  of  weapon  system's 
operational  effectiveness  and  suitability. 

32.  Instructional  System  Development  (ISD). 

33.  Integrated  Logistics  Support  (ILS).  The  process  that 
identifies  the  organic  Air  Force  Functions  required  to 
support  OSS  in  a  timely,  systematic,  and  orderly  manner. 

The  process  requires  continual  analysis  of  design  to 
determine  logistics  impacts  and  to  select  those  that 
minimize  logistics  support  burdens  on  the  O&S  commands,  and 
to  make  certain  that  logistics  support  is  available  at  the 
operating  location  upon  delivery  of  the  system  or  equipment 
for  use  U4:Vol  VII, 44). 

34.  Joint  Integrated  Avionics  Working  Group  (JIAWG). 

35.  MAJCOM.  Major  Command. 

36.  Milestone.  Major  program  decision  points  which  require 
Secretary  of  Defense  approval  to  proceed  with  the  weapon 
system  acquisition.  Milestone  .0  proceeds  CE/D  phase. 
Milestone  I  proceeds  D/V  phase.  Milestone  II  precedes  FSED 
(14:Vol  VII ,51) . 

37.  Mission  Area  Analysis  and  Operational  Cost 
Effectiveness  Analysis  (MAA/COEA) . 

38.  Mission  Need.  Deficiencies  (threat  changes,  decreased 
performance  of  older  systems,  or  national  security  policy 
changes ),  technol ogical  opportunity  or  expanded  mission 
which  created  the  need  for  a  new  or  modified  weapon  system 
(14:Vol  VII, 51) . 

39.  Mission  Element  Need  Statement  (MENS).  User-generated 
document  which  contains  information  on  mission  and  threat, 
alternative  concepts,  technology,  and  funding  implications 
of  the  weapon  system  development  (14:Vol  VII, 51). 

40 .  Memorandum  of  Agreement  (MOA) . 

41.  Manpower,  Personnel,  Training  and  Safety  (MPTS) . 

42.  Maintenance  Training  System  (MTS). 

43.  Naval  Air  Systems  Command  (NAVAIRSYSCOM) . 

44.  Organizational  Intermediate  (O&I). 
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45.  Operations  and  Support  (O&S) . 

46.  Office  of  Management  and  Budget  (0MB)  Circular 
A-109.  This  document  is  the  institutional  basis  for  major 
systems  acquisitions  for  all  executive  agencies.  It 
contains  management  principles  for  research,  development, 
and  acquisition  (14:Vol  VII, 37). 

47.  Operational  Requirements.  Requirements  specified  by 
the  user  and  support  command  including  the  Statement  of  Need 
(SON),  the  System  Operational  Requirements  Ooctunent  (SORD), 
and  Depot  Support  Requirements  Docximent 

(DSRD),  and  scenarios  (14:Vol  VII, 54). 

48.  Program  Management  Direction  (PMD).  Document  issued  by 
HQ  USAF  assigning  program  responsibility  to  Air  Force  field 
commands.  AFSC  Form  56  (issued  by  HQ  AFSC)  assigns  specific 
program  responsibility  to  AFSC  organizations. 

49.  Pre-Production  Verification  (PPV). 

50.  Required  Assets  Available  (RAA) . 

51.  Research  and  Development  (R&D). 

52.  Scenarios.  Complete  description  of  a  mission  including 
force  compositions,  targets,  mission  objectives,  and 
timeliness  (14:Vol  VII, 66). 

53.  Schedule.  Description  of  sequence  of  events  and  their 
relationships  for  the  development  of  a  weapon  system. 

54.  Statement  of  Need  (SON).  Document  generated  by  the 
user  which  identifies  the  initial  mission  need  in  general 
operational  terms.  Includes  a  Parametric  Spreadsheet  which 
describes  the  essential  characteristics  and 
quantitative/qualitative  performance  and  technical 
requirements  the  user  deems  relevant  to  the  mission  success. 
Also  includes  a  rational  and  requirements  change  sheet  which 
provides  ah  audit  trail  for  agreed-to  program  changes,  cost 
and  performance  tradeoffs,  growth  plans,  and  supporting 
rational  for  decisions  (14:Vol  VII, 69). 

55.  System  Operational  Requirements  Document  (SORD). 
Doctiment  generated  by  the  User  which  amplifies  and  refines 
on  the  SON  by  describing  pertinent  variables  associated  with 
candidate  system's  operation,  deployment,  employment, 
support,  training,  manpower,  etc..(14:Voi  VII, 69). 

56.  System  Program  Office  (SPO).  AFSC  divisional 
organization  that  is  responsible  for  a  weapon  system 
acquisition  program. 
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57.  Tactical  Air  Command  (TAG).  Major  Air  command 
responsible  for  tactical  air  operations  including 
counterair,  air  interdiction,  and  close  air  support. 

58.  Tactical  Air  Command  Liaison  Officer  (TACLO) .  TAG 
representative  collocated  in  the  SPO. 

59.  Tactical  Air  Force  (TAF) .  Consists  of  USAFE,  TAG, 

PACAF . 

60.  Test  and  Evaluation  Master  Plan  (TEMP).  An  overall 
test  and  evaluation  plan  designed  to  identify  and  integrate 
the  effort  and  schedules  of  all  T&E  to  be  accomplished 
within  an  acquisition  program.  Specifies  test  objectives, 
expected  methods  of  test,  data  requirements,  resource 
requirements,  and  data  reduction/analysis  techniques. 
Overall,  the  TEMP  states  how  tests  will  be  conducted  and  how 
these  results  will  be  used  to  verify  the  stated  requirements 
(14:Vol  VII, 78). 

61.  Test  Planning  Working  Group  (TPWG) .  Consists  of 
project  managers,  test  managers,  engineers,  pilots. 

Operation  and  Support  (O&S) ,  Manpower,  Personnel,  Training, 
and  Safety  (MPTS),  and  contractor  personnel  who  are 
responsible  for  providing  recommendations  to  the  program 
director  on  test  issues  (3;  4;  5;  11;  12). 

62.  Trained  Personnel  Required  (TPR) . 

63.  User.  End-user  of  the  system  being  acquired  (14:Vol 
VII  ,81) . 

64.  USAF  laboratories.  Organizations  responsible  for 
conducting  Air  Force  RSD  programs.  Include  Armstrong 
Aerospace  Medical  Research  Laboratories  (AAMRL),  Wright 
Research  and  Development  Center  (WRDC  Labs),  Munitions 
System  Division  (MSD),  and  Rome  Air  Development  Center 
(RADC)  (14:Vol  VII, 81) . 
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FOREWORD 


This  handbook,  the  product  of  a  comprehensive  study  of 
the  Advanced  Tactical  Fighter  Early,  Structured,  and 
Continuous  interface  process  with  the  user,  was  developed  to 
support  a  continuing  Command-wide  effort  to  improve  the 
process  for  acquiring  weapon  systems.  Its  goal  was  to 
document  the  ATF  ESC  user-interface  process  and  provide  a 
management  tool  to  other  Air  Force  acquisition  programs.  It 
was  originally  designed  to  assist  other  SPOs  in  developing 
and  maintaining  early,  structured,  and  continuous  interface 
with  the  user  throughout  the  Demonstration  and  Validation 
phase  of  an  acquisition  program. 

This  is  a  preliminary,  draft  copy  of  the  handbook.  It 
is  currently  being  coordinated  with  the  ATF  SPO.  It  will  be 
properly  reviewed  and  coordinated  when  a  final  copy  is 
completed,  and  it  is  determined  how  the  handbook  will  be 
used  by  the  Air  Force. 
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SECTION  1.0 
INTRODUCTION 

Historically,  the  user  has  inadvertently  been  excluded 
from  the  acquisition  process  after  the  approval  of  the 
Mission  Element  Need  Statement  (MENS),  or  became  proactive 
after  the  weapon  system  design  was  baselined.  Typically, 
therefore,  the  process  of  identifying  and  refining  weapon 
system  requirements  through  the  trade  study  process  is  not 
responsive  to  program  changes,  and  the  results  lack 
justification  and  traceability  because  SPOs  have  not 
developed  an  early,  structured,  and  continuous  interface 
with  the  user. 

The  ATF  ESC  SPO-User  team-building  handbook  discusses 
specific  management  and  engineering  activities  that  were 
implemented  by  the  ATF  SPO  in  response  to  a  need  for  a 
better  relationship  between  the  SPO  and  the  user  during  the 
acquisition  process.  Currently,  it  is  applicable  to  the 
Demonstration  and  Validation  phase  of  major  acquisition 
programs.  The  acquisition  process  is  presented  from  an  ATF 
SPO  viewpoint  including:  roles  and  responsibilities  of 
Government  and  contractors;  documentation  generated  by  a 
SPO;  objectives  of  the  Demonstration  and  Validation  phase. 

This  version  of  the  ATF  ESC  handbook  is  a  preliminary, 
draft  document.  It  has  not  been  fully  coordinated  with  the 
ATF  SPO . 

Any  questions  or  comments  concerning  the  ATF  ESC 
handbook  should  be  directed  to  Captain  Robert  K.  Barry, 
AFIT/LSG,  Wright-Patterson  AFB  OH,  ADTOVON  785-8989;  or. 
Major  Janet  Bloom,  ASD/YFM,  Wright-Patterson  AFB  OH,  AUTOVON 
785-1422. 
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SECTION  2.0 
ASSUMPTIONS 


The  handbook  was  designed  as  a  management  tool  for 
integrating  the  user  into  the  acquisition  process. 
Consideration  should  be  given  to  the  following  assumptions 
when  planning  and  implementing  a  user-interface  process 
based  on  some  of  the  ATF  concepts. 

1.  Based  on  draft  Defense  Management  Review  (DMR) 
directions,  a  System  Program  Office  will  not  be  formed  until 
after  Milestone  I  (15). 

2.  User  deficiencies/needs  were  validated  by  the 
Milestone  0  decision  (15). 

3.  Concept  Exploration  and  Definition  funding  pool  was 
available  to  fund  the  concept  direction  studies  and 
development  of  alternative  technical  solutions  (15). 

4.  The  primary  participants  in  the  ATF  ESC  process 
were  the  System  Program  Office,  user,  contractor,  and  AFSC. 

5.  The  customers  of  process  were  the  user,  DAB  II 
process,  D/V  program  directors.  Science  Advisory  Boards 
(SAB),  System  Design  Reviews  (SDRs),  Mission  Area 
Analysis/Cost  and  Operational  Effectiveness  Analysis 
(MAA/COEAS),  program  reviews.  Technical  Interchange  Meetings 
(TIMs),  and  working  groups. 

6.  The  ATF  ESC  process  was  planned  and  implemented  in 
the  Demonstration  and  Validation  phase. 

7.  Major  programs  were  addressed,  but  the  philosophy 
may  be  applicable  to  less-than  major  programs. 

8.  The  System  Program  Office  specified  the 
requirements  in  the  form  of  a  threat  description  and 
preliminary  weapon  system  specification. 

9.  The  acquisition  strategy  involved  contractor 
teaming  arrangements,  competition,  and  prototyping. 
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SECTION  3.0 
APEX  TEAM 

General  Randolph,  former  commander  of  Air  Force  Systems 
Command  (AFSC) ,  spearheaded  an  initiative  in  1989  that 
sought  out  ways  to  improve  the  current  acquisition  process. 
Major  changes  were  required  because  of  today's  environment 
of  declining  resources  and  clear  emphasis  on  technological 
solutions  to  our  nation's  defense  needs  (15;  39).  This 
initiative  later  became  known  as  APEX.  The  mission  of  APEX 
was  to  create  a  revolutionary  team  acquisition  process  that 
destroyed  barriers,  inspired  opportunity  for  improvement. 

Hundreds  of  interviews  were  conducted  by  APEX  team 
members  to  identify  no- value-added  tasks  associated  with  the 
acquisition  of  weapon  systems  which  could  or  should  be 
deleted  or  improved  (15).  Program  directors,  project 
managers,  industry,  users,  functional  organization 
personnel,  and  acquisition  executives  from  other  services 
were  interviewed  by  the  team  interviewees  (15).  In  addition 
to  the  perceived  problems  and  problem  areas,  more  than  300 
improvement  ideas  were  identified. 

The  APEX  team  concluded,  in  part,  that  SPOs  needed  to 
develop  a  better  interface  process  with  their  participants, 
including  the  user,  particularly  in  the  early  phases  of  the 
acquisition  process  (15).  Also,  the  study  identified  a 
number  of  concern  areas  associated  with  the  requirements 
generation  process  which  it  felt  every  SPO-user  team¬ 
building  strategy  should  consider  (16): 

a.  Good  Cooperation  and  Exchange  of  Ideas.  The 
program  director  and  his  user  counterpart  must  create  an 
environment  of  openness  and  trust.  They  must  be  committed 
to  the  establishment  of  an  effective  SPO-user  team  process. 
They  must  also  ensure  that  the  personnel  responsible  for 
implementing  the  user-based  interface  strategy  have  the 
necessary  guidance.  The  user  must  be  integrated  into  the 
acquisition  strategy,  and  be  involved  in  every  aspect  of  the 
program. 

b.  Disciplined  Requirements  Generation.  The  user 
believes  the  requirements  definition  process  should  be 
separated  from  the  Planning  Programming  and  Budgeting  System 
(PPBS)  process.  The  requirements  process  is  set  up  to  make 
informed  cost-schedule-performance-tradeoffs  at  critical 
points  during  the  acquisition  process.  Whereas,  the  PPBS 
typically  is  not  set  up  to  make  informed  affordability 
assessments  and  resource  allocation  decisions  on  defense 
acquisition  programs.  Moreover,  SPO  analysis  techniques  do 
not  adequately  reflect  cost  and  schedule  constraints.  Also, 
the  user  feels  that  AFSC  needs  to  commit  to  Preplanned 
Product  Improvement  (P3I)  programs  earlier  in  the 
acquisition  cycle. 

c.  Team  Approach.  The  SPO  user-interface  strategy 
should  consider  a  team  acquisition  strategy  throughout  the 
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life  of  the  program.  The  user  should  be  involved  in  the 
day-to-day  decisions.  The  team  should  operate  with  complete 
candor.  The  user  also  needs  to  be  better  educated  in  the 
acquisition  process. 

d.  Mission  Area  Analysis/Cost  of  Operational 
Effectiveness  Analysis  (MAA/COEA) .  Historically,  AFSC  has 
not  committed  the  necessary  resources  to  execute  this 
analysis.  Moreover,  the  user  has  been  excluded  from  this 
analysis . 

e.  Concept  DIR  Studies.  This  concern  is  not 
applicable  to  this  study. 

f.  Approach  to  Major  and  Non-Major  Programs.  This 
concern  is  not  applicable  to  this  study. 

g.  Exclusion  from  Acquisition  Process.  Typically, 
when  the  operational  needs  are  identified,  the  user  does  not 
remain  a  responsible  and  useful  partner  in  the  development 
and  acquisition  process.  Ideally,  the  user  must  be  involved 
in  every  program  action  to  ensure  the  TAP  is  provided  with 
the  best  operational  capabilities  (39).  Otherwise,  the 
requirements  process  will  not  be  as  effective,  and  the 
contributions  by  the  user  will  be  less  than  optimal  (39). 

h.  Improved  Synergism  of  Documentation.  A  strategic 
plan  needs  to  be  developed  up  front  to  synergise  all 
applicable  program  documentation.  This  includes  clear 
system  definition,  a  threat  baseline,  and  standard 
simulations.  It  is  critical  that  the  system  specification 
can  be  traced  back  to  the  system  requirements  and  the 
Operational  Requirements  Docvunent  (ORD)  through  the  trade 
study  process.  Also,  the  ORD  should  be  updated  after  every 
major  trade  or  program  action  rather  than  at  every  major 
milestone.  All  this  is  key  to  having  accurate  and  timely 
information  available  when  it  is  needed. 
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SECTION  4.0 
PROGRAM  SUMMARY 


4.1  ATF  PROGRAM  OVERVIEW.  The  ATF  is  being  developed  as  an 
Air  Superiority  fighter  to  dominate  the  threat  in  the  year 
2000  and  beyond  with  a  primary  design  mission  of  offensive 
counterair  (13).  Developing  a  system  with  improved 
performance  characteristics,  such  as  supersonic  persistency, 
high  maneuverability,  and  better  combat  radius,  within 
congressional ly  mandated  goals  for  a  50,000  pound  aircraft 
at  35  million  dollar  average  unit  flyaway  costs  (base  year 
1985  dollars),  was  a  challenging  task  requiring  optimization 
of  numerous  variables  which  had  to  be  treated  in  an 
integrated  analysis  (13). 

The  program  just  completed  a  competitive  Demonstration 
and  Validation  (D/V)  phase.  During  the  D/V  phase,  the  most 
promising  technical  alternatives  from  the  Conceptual 
Exploration/Demonstration  (CE/D)  are  refined  by  making 
performance  tradeoffs  whereby  high-cost  features  providing 
only  marginal  performance  gains  are  deleted  from  the  system 
(37:301).  It  will  enter  Full  Scale  Development  (FSD)  in 
mid-1990.  Production  options  are  scheduled  to  be  exercised 
beginning  in  1996  with  the  Tactical  Air  Force  receiving 
aircraft  beginning  in  1999. 

4.2  SYSTEM  DEFINITION  AND  DESIGN.  The  ATF  was  designed 
primarily  for  a  single  mission  requirement-offensive 
counterair.  The  SPO  director  pushed  the  use  of  state-of- 
the-art  technology  and  discriminate  application  and 
tailoring  of  military  standards  and  specifications  (30).  He 
also  desired  100  percent  commonality  and  interoperability 
with  existing  O&S  equipment.  The  trade  study  process  which 
followed  required  extensive  interaction  between  the 
technical  people  and  the  User  in  order  to  match  requirements 
with  achievable  technologies  (1). 

4.3  TRADE  STUDY  APPROACH.  The  process  will  be  described 
later  on  in  the  handbook.  The  process  resulted  in  a  match 
of  requirements  and  available  technologies  to  baseline  an 
executable  program  in  FSED  (13).  This  led  to  an 
unprecedented  level  of  technical  understanding  of  the 
requirements  and  traceability  to  the  original  requirements. 
Cost  was  factored  into  requirements  trades.  This  process 
identified  the  requirements  that  had  to  be  flight  tested  in 
D/V.  The  program  will  enter  FSD  with  low  to  moderate 
program  risk  in  all  areas.  The  TAG  Commander  reviewed  the 
status  of  the  contractor  trade  studies  on  an  annual  basis. 
This  review  required  unparalleled  teamwork  between  the  SPO, 
user,  contractor,  and  other  government  organizations.  In 
short,  the  user  had  a  willingness  to  adjust  his  requirements 
based  on  the  contractor  trade  studies  which  demonstrated 
technical  feasibility. 

4.4  TAF  REQUIREMENTS  PROCESS.  The  ATF  ESC  process  was 
implemented  without  changing  the  roles  and  responsibilities 
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of  the  Supporting,  Operating,  and  Implementing  commands. 

TAP  management  responsibility  for  acquisition  programs  also 
remained  the  same.  This  TAP  management  responsibility  is 
summarized  below. 

4.4.1  REQUIREMENTS  MANAGEMENT.  TAC/DR  maintains  close 
contact  throughout  the  entire  acquisition  process  from 
program  initiation  to  production  and  deployment  and 
operational  support  (43).  TAC/DR  is  subdivided  into  six 
directorates  and  three  major  Special  Management 
Organizations  (SMOs)  to  work  and  manage  programs.  Liaison 
offices  were  established  to  assist  TAC/DR  in  its  management 
function.  DR  has  eight  liaison  offices  located  at  selected 
Air  Porce  activities,  primarily  APSC  divisions.  They  serve 
as  the  TAP  representatives  and  play  a  major  role  in  their 
management  of  TAP  requirements  during  the  acquisition 
process  (43). 

4.4.2  TAP/TAC  RELATIONSHIP.  TAC/DR  is  designated  the 
requirements  spokesman  for  the  TAP  (TAC,  USAPE,  PACAP) 
throughout  the  development  and  acquisition  process  (43). 

They  perform  the  administrative  functions  of  coordination, 
justification,  and  presentation  of  the  TAP  requirements 
(43). 

4.4.3  POM/BES  INVOLVEMENT.  Another  key  aspect  of  TAC's 
involvement  in  the  requirements  process  is  their  attempt  to 
influence  the  formulation  of  the  POM/BES  to  ensure  that 
TAP's  most  urgent  needs  are  funded  (43).  TAC  POM  working, 
groups,  with  DR  representatives  and/or  chairmen,  are 
critical  elements  in  the  efficient  and  effective  production 
of  a  prioritized  listing  of  TAP  RD&A  programs  for  submission 
to  Air  Staff  for  their  use  during  POM  formulation  (43). 

4.4.4  TESTING  RESPONSIBILITY.  Testing  is  another  vital 
ingredient  in  a  successful  acquisition  program.  There  are 
two  types  of  testing  associated  with  acquisition: 

Development  Test  and  Evaluation  (DT&E)  and  Operational  Test 
and  Evaluation  (OT&E).  DT&E  is  normally  performed  by  APSC. 
TAC/DR  is  the  responsible  staff  agency  for  all  OT&E 
conducted  within  the  command. 

4.4.5  RELATIONSHIP  WITH  APSC.  TAC/DR  is  deeply  involved 
with  APSC  because  they  are  responsible  for  the  development 
and  acquisition  of  weapon  systems  and  support  equipment  that 
are  responsive  to  the  needs  of  the  TAP  (43: Chapter  5,17). 

It  is  important  that  TAC/DR  and  APSC  work  closely  throughout 
the  acquisition  process. 

4.4.6  RELATIONSHIP  WITH  APLC.  TAC/DR  is  also  involved  with 
APLC  on  programs  which  modify  inventory  systems  and  support 
equipment  (43: Chapter  5,18). 

4.4.7  RELATIONSHIP  WITH  THE  CONTRACTOR.  TAC/DR  has 
frequent  contacts  with  contractor  representatives,  and  it  is 
important  that  they  understand  what  the  Air  Porce  expects 
that  relationship  to  be  (43:Chapter  5,19).  The  User  was 
allowed  to  meet  with  the  contractors  without  SPO 
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representatives  present.  However,  they  were  required  to 
send  visit  notifications  for  each  visit. 

4.5  THREAT .  The  program  director  ensured  that  the 
Government  and  the  Contractor  were  using  a  baselined  threat 
description.  He  ensured  that  the  user  was  involved  in 
baselining  the  threat  description.  He  also  ensured  that  the 
updates  were  properly  identified,  evaluated,  coordinated, 
and  incorporated  into  the  threat  description.  In  order  to 
accomplish  this,  the  SPO  establish  a  Threat  Working  Group 
(TWG)  to  coordinate  this  process. 

4.6  TEST  PHILOSOPHY.  The  objective  of  the  trade  study 
process  was  to  match  requirements  with  achievable 
technologies  (1).  Analysis  and  test  played  a  crucial  role 
in  defining  and  documenting  these  technologies  as  well  as 
the  potential  Pre-planned  Product  Improvements  (PPPI).  Key 
concepts  associated  with  the  ATP  test  philosophy  were  as 
foil ows : 

4.6.1  TEAM  CONCEPT.  The  test  manager  ensured  that  the 
User,  and  the  Contractor,  O&S,  MPTS,  AFOTEC,  flight  test 
center,  and  ALD  communities  were  involved  in  PMR,  worki."* 
group,  test  planning,  and  technical  meetings. 

4.6.2  TEMP .  The  test  manager  ensured  that  the  User,  and 
the  Contractor,  O&S,  MPTS,  AFOTEC,  flight  test  center,  and 
ALD  communities  were  involved  in  the  preparation  and  review 
of  the  TEMP. 

4.6.3  CLASS  III  MOCKUPS.  Since  the  prototypes  were  not 
available  until  late  in  the  D/V  program,  the  contractor 
built  Class  III  mockups  for  supportability  and 
maintainability  demonstrations  as  well  as  the  MPTS  efforts 
(38). 

4.6.4  ”FLY-BEFORE-BUY” .  The  SPO  employed  the  "fly-before¬ 
buy"  approach  by  testing  with  prototypes  before  committing 
funds  to  FSED. 

4.6.5  AVIONICS  FLYING  LAB.  The  contractors  used  Avionics 
Flying  Lab  (AFL)  to  test  avionics  design  concepts  not 
planned  to  be  tested  on  the  prototypes.  It  also 
familiarized  the  pilots  with  the  ATF  avionics  sweet. 

4.6.6  GROUND  TESTING.  The  contractor  used  ground  testing 
to  test  systems,  subsystems,  and  components  that  would  not 
be  tested  on  the  prototype.  Ground  testing  was  also  used  to 
support  flight  testing,  especially  for  the  high-risk  flight 
tests . 

4.7  ORGANIZATION.  The  ATF  was  a  very  political  program. 

It  was  also  very  important  to  the  Air  Force.  It  involved  a 
large  number  of  DoD  and  contractor  organizations.  The  ATF 
SPO  employed  unique  organizational  techniques  to  maintain 
control  of  the  program  and  the  funding,  and  to  remain  a 
credible  counterpart  to  the  user.  These  techniques  were 
discussed  briefly  below: 

4.7.1  USER  REPRESENTATIVES.  User  representatives  from  ATC, 
AFLC,  and  TAC  were  collocated  in  the  ATF  SPO. 
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4.7.2  O&S/MPTS  EXPERTS.  The  DPML  attempted  to  acquire 
maintainability  experts  for  each  opreational  AFSC  (30;  33; 
38).  The  program  director  should  also  established  a  team  of 
training  experts  within  the  SPO  which  was  made  up  of  ASD/RW 
and  ATC  representatives  (41). 

4.7.3  EXPERIENCED  SPO  PERSONNEL  Personnel,  pilots, 
trainers,  and  maintainers,  with  current  and  extensive 
backgrounds  with  the  weapon  system  being  procured,  should 
occupy  key  engineering  and  management  positions  in  the  SPO. 

4.7.4  ENGINE  RESPONSIBILITY.  The  engine  program  is, 
typically,  a  separate  program  from  the  weapon  system. 
However,  the  ATF  SPO  controlled  the  engine  program  which  is 
normally  managed  by  Deputy  for  Propulsion,  Aeronautical 
Systems  Division,  ASD/YZ  in  other  aircraft  programs.  These 
personnel  were  located  in  the  ATF  SPO  and  reported  directly 
to  the  Program  Director.  This  approach  gave  the  Program 
Director  complete  control  of  the  participating  organizations 
during  D/V. 

4.7.5  AVIONICS  SUPPORT.  The  ATF  SPO  controlled  the 
technical  support  out  of  Deputy  for  Reconnaissance  and 
Electronic  Warfare,  Aeronautical  Systems  Division,  ASD/RW, 
which  is  normally  collocated  from  ASD/RW.  ASD/RW  provides 
avionics  technical  support.  These  personnel  were  located  in 
the  ATP  SPO  and  reported  to  the  ATF  program  director. 

4.7.6  LABORATORY  TECHNOLOGY.  The  ATP  SPO  siped 
Memorandums  of  Agreement  (MOA)  with  all  the  Air  Force 
laboratories.  They  established  and  controlled  technology 
panels  which  met  on  a  periodic  basis  and  involved  AFSC,  AFSC 
product  divisions,  Air  Force  laboratories.  User,  ALDs,  and 
flight  test  communities.  The  purpose  of  these  panels  was  to 
identify  any  technology  that  may  be  applicable  to  the  ATF. 
The  SPO  also  established  a  technology  transition  working 
group  with  the  contractor  which  piggy-backed  onto  the  trade 
study  process. 

4.7.7  ARMAMENT  I NTEGRAT I ON .  Munitions  System  Division 
(MSD)  established  an  operating  location  within  the  ATF  SPO 
to  ensure  a  balanced  weapon  system  design.  A  Memorandum  of 
Agreement  specified  the  required  support.  The  MSD 
representative  provided  baseline  weapons  data  to  ensure 
total  weapon  system  integration,  and  ensured  armament 
program  cost,  schedule,  technical,  and  program  direction  was 
consistent  with  the  ATF  program.  The  ATF  SPO  prepared  a 
Weapons  Integration  Plan  which  was  a  major  tool  of  the 
Armament  Integration  Working  Group,  and  ensured  the  ATF 
design  was  a  weaponized  design. 

4.7.8  TRAINING  PLANNING  TEAM.  SPOs  normally  rely  on  MPTS 
support  from  the  Training  Advisory  Office.  On  the  other 
hand,  the  ATF  SPO  established  a  team  of  MPTS  experts  and 
collocated  them  in  the  their  SPO  (41).  They  established  a 
Training  Planning  Team  (TPT)  to  ensure  that  MPTS 
requirements  were  considered  in  the  design  process.  The  TPT 
will  be  discussed  later  on  in  the  handbook  (41). 
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4.7.9  MAINTENANCE  TEAM.  Refer  to  Section  8.0  for 
discussion  on  this  concept. 

4.8  INTEGRATED  DESIGN  APPROACH.  Typically,  OSS  and  MPTS 
requirements  are  considered  only  after  the  system  design  has 
been  frozen.  That  is,  the  experts  in  this  area  have  to  use 
the  results  of  the  design  process  to  find  a  supportability 
and  MPTS  answer  to  a  fixed  design  (13).  Another  objective 
of  ATF  D/V  approach  was  to  force  early  integration  of  OSS 
and  MPTS  requirements  into  the  weapon  system  design,  and 
deliver  a  plan  which  ensured  these  requirements  were  met 
(13).  The  government  process  includes  assembling  a  team 
with  MPTS  experts  within  the  SPO  to  work  MPTS  requirements. 
This  MPTS  team  can  be  extended  to  Training  Planning  Teams 
(TPT),  expanding  the  scope  of  TPTs  and  providing  a  forum  for 
MPTS  discussions  (13).  The  SPO  had  the  same  philosophy  and 
approach  in  considering  O&S  requirements.  This  review  did 
not  address  the  corresponding  contractor  process  and  the 
relationship  between  the  Government  and  Contractor  process. 

4.9  CONTRACTING  APPROACH.  The  ATP  program  was  a  high-risk 
program.  The  system  design  approaches  used  state-of-the-art 
technology  that  required  advanced  manufacturing  techniques. 
ATF  SPO  management  directed  its  personnel  to  push  Air  Force 
policy  and  guidance  to  accommodate  the  ATF  program.  DoD 
also  required  each  contractor  to  invest  some  of  their  own 
funds  in  the  program  due  to  the  nature  of  the  technologies. 
The  contractors  shared  a  greater  percentage  of  the  risk  than 
on  other  D/V  programs.  Therefore,  Firm  Fixed  Price 
contracts  were  let.  The  approach  also  involved  an  extended 
D/V  phase,  contractor  teaming  arrangements,  competition,  and 
prototyping. 
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SECTION  5.0 

ATF  ESC  USER- INTERFACE  PROCESS 

The  ATF  SPO  developed  the  ESC  user-interface  process  in 
response  to  a  need  for  a  better  relationship  between  the  SPO 
and  the  user  in  their  D/V  phase.  The  relationship  was 
critical  during  this  phase  because  the  SPO  and  the  user  had 
the  greatest  opportunity  to  change  the  weapon  system  design 
before  it  was  baselined  in  Full  Scale  Engineering 
Development  (FSED)  (23).  The  primary  objectives  of  the  ATF 
ESC  process  were  to  add  justification  and  accountability  to 
the  requirements  process  by  involving  and  educating  the  user 
during  the  trade  studies  process,  and  to  ensure  their 
continued  support  in  a  very  political  program. 

The  user  was  involved  in  every  aspect  of  the  program  to 
ensure  the  operational  requirements  became  an  integral  part 
of  the  weapon  system  design  process.  The  degree  of 
interaction  between  the  User  and  the  engineering  communities 
was  unparalleled.  Representatives  from  the  TAP,  Air  Force 
Logistics  Command  (AFLC) ,  and  Air  Training  Command  (ATC), 
for  the  ATF,  were  collocated  in  the  program  office  and 
treated  as  members  of  the  SPO.  Formal  agreements,  such  a 
Memorandum  of  Agreements  (MOAs)  were  needed  due  to  the 
program's  support  from  senior  Air  Force  management.  The  ATF 
SPO  established  a  team  of  experts  for  the  O&S  and  MPTS 
functions  within  the  SPO  to  ensure  that  they  were  integrated 
into  the  system  design.  The  user  attended  working  groups, 
Technical  Interchange  Meetings  (TIMs),  program  reviews,  and 
briefings  at  all  levels.  The  user  participated  in  Mission 
Area  Analysis/Cost  and  Operational  Effectiveness  Analysis 
(MAA/COEAs)  and  other  ATF-related  studies. 
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SECTION  6.0 
REFERENCE  DOCUMENTS 

6.0  REFERENCE  DOCUMENTS. 

6.1  BACKGROUND  DOCUMENTS. 

a.  ATF  System  Training  Plan  (STP),  14  Jan  1991. 

b.  Draft  Concept  Of  Operations  ATP  Integrated 
Maintenance  System  (AIMS),  15  Aug  1990. 

c.  ATP  Armament  Integration  Plan,  24  Feb  87. 

d.  Memorandum  of  Agreement  (MOA)  between  ASD/YF  AND 
MSD/CV,  26  April  88. 

e.  Sample  Technology  Fact  Sheet,  AFATL,  Eglin  AFB  FL, 
13  Sep  88 

6.2  OTHER  APPLICABLE  DOCUMENTS. 

a.  ATF  Armament  Working  Group  Charter, 

b.  ATF  Avionics  Working  Group  Charter, 

c.  ATF  Threat  Working  Group  Charter, 

d.  ATF  Cockpit  Working  Group  Charter, 

e.  ATF  Test  Planning  Working  Group  Charter, 
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SECTION  7.0 

GOOD  COOPERATION  AND  EXCHANGE  OF  IDEAS 


7.0  The  following  concepts  were  used  by  the  ATF  SPO  to 
ensure  good  cooperation  and  exchange  of  ideas  with  the  user. 
The  program  director  may  want  to  consider  these  concepts 
when  planning  his/her  D/V  strategy. 

7.1  MANAGEMENT  COMMITMENT.  The  commitment  by  senior 
management  to  execute  the  ATF  system  definition  and  design 
concepts,  and  integrate  the  User,  early,  in  the  acquisition 
process  was  crucial  to  the  success  of  the  ATF  ESC  user- 
interface  process  (1).  The  Air  Force  Chief  of  Staff,  TAC 
Commander,  AFSC  Commander,  ALD  Commander,  and  SPO  director 
were  all  committed  to  creating  an  environment  of  openness 
and  trust.  The  SPO  director  and  his  User  counterparts  also 
managed  to  create  an  environment  of  openness  and  trust 
because  of  their  personalities  and  innovative  management 
techniques.  The  SPO  director  may  find  some  of  the  related 
concepts  useful  to  his/her  program. 

7.2  EARLY  USER  INVOLVEMENT.  Based  on  lessons  learned  from 
other  programs,  it  is  easier  and  less  costly  to  integrate 
manufacturing,  supportablility ,  maintainability,  and  cost 
into  the  initial  design  while  it  is  evolving  (1).  This,  can 
avoid  costly  weapon  system  modifications  and  upgrades  at  a 
later  date.  In  order  to  accomplish  this  the  SPO  should 
recruit  program  managers,  engineers,  and  maintainers  with 
current  backgrounds  in  the  weapon  system  being  acquired. 

They  should  be  placed  in  key  management  and  engineering 
positions.  User  representatives  should  also  be  brought  into 
the  SPO  at  the  beginning  of  D/V.  The  maintenance  personnel 
should  be  given  the  latitude  (including  TDY  funding)  by 
management  to  acquire  and  use  any  expert  within  DoD  and  the 
defense  contractors  in  the  required  discipline  areas,  such 
as  AIMS,  IMIS,  and  MATE  (30;  33;  38).  Also,  User  and 
experts  participation  in  PMRs ,  working  groups,  and  technical 
meetings  is  part  of  education  process,  and  is  crucial  to  a 
program's  success. 

7.3  ORGAN I Z AT I ON .  The  ATF  SPO  employed  some  unique 
organizational  techniques  to  maintain  control  of  the  program 
and  the  funding,  and  to  remain  a  credible  counterpart  to  the 
user.  These  techniques  were: 

7.3.1  USER  REPRESENTATIVES.  User  representatives  from  ATC, 
AFLC,  and  TAC  should  be  collocated  in  the  ATF  SPO. 

7.3.2  O&S/MPTS  EXPERTS.  The  DPML  should  attempt  to 
acquire  maintainability  experts  for  each  operational  AFSC 
(30;  33;  38).  The  SPO  director  should  assemble  a  team  of 
training  experts  within  the  SPO.  The  team  should  consist  of 
ASD/RW,  Training  System  Office  (TSO) ,  engineers,  and  ATC 
User-representatives . 

7.3.3  EXPERIENCED  SPO  PERSONNEL  Personnel,  suae  as 
pilots,  trainers,  and  maintainers,  with  current  and 
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extensive  background  in  weapon  system  being  procured,  should 
occupy  key  engineering  and  management  positions  in  the  SPO. 

7.3.4  ENGINE  RESPONSIBILITY.  The  engine  program  is 
generally  a  separate  program  from  the  weapon  system. 

However,  the  SPO  should  control  the  engine  program  which  is 
normally  managed  by  Deputy  for  Propulsion,  Aeronautical 
Systems  Division,  ASD/YZ  in  other  aircraft  programs.  The 
personnel  should  be  located  in  the  SPO  and  report  directly 
to  the  Program  Director. 

7.3.5  AVIONICS  SUPPORT.  The  SPO  should  also  control  the 
technical  support  out  of  Deputy  for  Reconnaissance  and 
Electronic  Warfare,  Aeronautical  Systems  Division,  ASD/RW, 
which  is  normally  collocated  from  ASD/RW.  ASD/RW  provides 
avionics  technical  support.  The  personnel  should  be  located 
in  the  SPO  and  report  to  the  program  director. 

7.3.6  LABORATORY  TECHNOLOGY.  The  SPO  should  sign 
Memorandums  of  Agreement  (MOA)  with  all  the  Air  Force 
laboratories.  They  should  establish  and  control  technology 
panels.  These  panels  should  meet  on  a  periodic  basis  and 
involve  APSC,  APSC  product  divisions.  Air  Force 
laboratories.  User,  ALDs,  and  flight  test  communities.  The 
purpose  of  these  panels  should  be  to  identify  any  technology 
that  may  be  applicable  to  the  system.  The  SPO  should  also 
establish  a  technology  transition  working  group  with  the 
contractor  to  ensure  the  contractor  is  not  duplicating  the 
efforts  of  the  Air  Force  laboratories,  and  that  he  is 
informed  about  technologies  that  could  be  used  on  the  weapon 
system.  The  contractor  should  not  use  the  lab  efforts  to 
justify  the  incorporation  of  technologies  into  the  baseline 
through  the  trade  study  process. 

7.3.7  ARMAMENT  I NTEGRAT I ON .  Munitions  System  Division 
(MSD)  should  establish  an  operating  location  within  the  SPO 
to  ensure  a  balanced  weapon  system  design.  A  Memorandum  of 
Agreement  should  be  written  to  specify  the  required  support. 
The  MSD  representative  can  be  useful  in  providing  baseline 
weapons  data  to  ensure  total  weapon  system  integration,  and 
ensured  armament  program  cost,  schedule,  technical,  and 
program  direction  was  consistent  with  the  program. 

The  SPO  should  also  prepare  a  Weapons  Integration  Plan  which 
should  be  a  major  tool  of  the  Armament  Integration  Working 
Group.  The  objective  of  the  ARWG  should  be  to  ensure  the 
weapon  system  design  is  a  weaponized  design. 

7.3.8  TRAINING  PLANNING  TEAM.  SPOs  normally  rely  on  MPTS 
support  from  the  Training  Advisory  Office.  Based  on  the  ATF 
program,  however,  the  SPO  should  establish  a  team  of  MPTS 
experts  and  collocate  them  in  the  their  SPO  (41).  The  team 
should  establish  a  Training  Planning  Team  (TPT)  to  ensure 
that  MPTS  requirements  are  considered  in  the  design  process. 
The  TPT  will  be  discussed  later  on  in  the  handbook  (41). 

7.3.9  MAINTENANCE  TEAM.  Refer  to  Section  8.0  for  more 
information  on  the  Maintenance  Team. 
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7.4  USER  PERSONNEL  COLLOCATED  IN  SPO.  Although  the  primary 
interface  should  be  through  TAC/ORB,  collocating  the  User 
personnel  in  the  SPO  can  be  helpful  to  the  SPO  when  TAC/DRB 
could  not  be  reached.  They  can  also  provide  continuity 
between  the  User  and  their  SPO  counterparts.  They  can  open 
up  the  lines  of  communications,  and  ensure  that  the  experts 
get  involved.  Specific  roles  and  responsibilities  of  the 
User  representatives  that  proved  to  be  beneficial  to  the  ATP 
SPO  as  well  as  the  User  are  discussed,  separately,  below. 
7.4.1  TACLO .  The  TACLO  should  be  totally  integrated  into 
the  SPO  activities.  He  should  attend  staff  meetings,  PMRs , 
working  groups,  technical  meetings,  and  briefings.  He 
should  be  was  allowed  to  participate  in  MAA/COEAs,  SABs,  and 
special  studies  as  required.  He  should  (20;  21;  23;  35; 

42): 

a.  Talk  to  TAC/DRB  daily. 

b.  Go  TDY  to  TAC/DR  once  a  month. 

c.  Write  and  coordinate  trip  reports  as  required  with 
the  SPO  director,  TAC/DRB,  and  other  SPO  personnel. 

d.  Be  credible.  Credibility  with  the  fighter 
community  was  extremely  important  to  generate  an  environment 
of  openness  and  trust.  Mutual  trust  results  in  open  and 
honest  exchange  of  even  privileged  information.  Mutual 
trust  also  ensures  that  there  are  no  surprises. 

e.  Have  direct  line  to  TAC/DR. 

f.  Be  in  place  before  the  start  of  D/V. 

g.  Depending  on  areas  of  expertise,  co-chair  some  of 
the  working  groups,  such  as  the  Scenario,  Threat,  and 
Cockpit  Working  Groups. 

h.  Be  a  member  of  Source  Selection  Evaluation  Board 
(SSEB) .  ■ 

i.  Be  cleared  to  all  programs  at  all  levels. 

j.  Be  allowed  to  attend  SPO  staff  meetings. 

k.  Be  required  to  attend  all  meetings,  briefings, 
working  groups,  and  Technical  Interchange  Meetings  (TIMs). 

l.  Be  allowed  to  participate  in  MAA/COEAs,  Multiple 
Aircraft  Reviews  (MARs),  Science  Advisory  Board  (SAB) 
Reviews,  and  special  studies,  as  required. 

7.4.2  ATC  User  Representatives.  The  ATC  User- 
representative  should  be  totally  integrated  into  the  SPO 
activities.  He  should  attend  staff  meetings,  PMRs,  working 
groups,  technical  meetings,  and  briefings.  The  roles  and 
responsibilities  of  this  ATC  representative  should  be  to 
(41): 

a.  Talk  to  ATC  headquarters  or  ASD  Training  Advisory 
Office  daily. 

b.  Write  activity  reports  once  a  month.  These  reports 
should  be  reviewed  by  the  SPO  director,  TPT  team  leader,  and 
Training  Advisory  Office.  Periodically,  these  reports  may 
need  to  be  sent  to  ATC  headquarters. 
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c.  Attend  staff  meetings,  PMRs,  working  groups, 
technical  meetings,  and  briefings.  He  should  participate  in 
MAA/COEAs,  SABs,  and  special  studies  as  required. 

d.  Be  credible.  Credibility  with  the  training 
community  was  extremely  important  to  generate  an  environment 
of  openness  and  trust. 

e.  Assist  the  team  in  setting  up  a  TPT,  which  will  be 
discussed  shortly. 

f.  Assist  in  the  development  of  a  training  plan  which 
should  document  how  the  team  will  ensure  that  the  training 
concept  developed  and  integrated  into  the  design. 

7.4.3  AFLC  User  Representatives.  The  DPML,  with  support 
from  the  SPO  director,  should  attempt  to  assign  a 
maintainability  specialist  for  each  operational  AFSC.  In 
addition,  he/she  should  assign  user  representatives  for  each 
AFSC,  or  at  least  in  the  AFSCs  that  he  is  unable  to  fill 
with  his/her  own  personnel .  The  O&S  User-representatives 
should  be  totally  integrated  into  the  SPO  activities.  They 
should  attend  staff  meetings,  PMRs,  working  groups, 
technical  meetings,  and  briefings.  The  roles  and 
responsibilities  of  these  User  representatives  should  be 
(23;  30;  31;  33;  38;  41): 

-a.  Talk  to  TAC/DRB,  ALDs ,  and  other  supporting 
agencies  daily. 

b.  Go  TDY  to  TAC/DR  once  a  month. 

c.  Write  and  coordinate  trip  reports  as  required  with 
the  SPO  director,  TAC/DRB,  and  other  SPO  personnel. 

d.  Be  credible.  Credibility  with  the  fighter 
community  was  extremely  important  to  generate  an  environment 
of  openness  and  trust.  Mutual  trust  results  in  open  and 
honest  exchange  of  even  privileged  information.  Mutual 
trust  also  ensures  that  there  are  no  surprises. 

e.  Be  in  place  before  the  start  of  D/V. 

f.  Establish  a  Maintainability  Team  (MT). 

Engineering,  TAC/DRBL,  TAC/DRB,  AFFTC,  ATC,  AFOTEC, 
maintainability  specialists,  ILS  specialists,  contractors, 
and  other  agency  experts  should  be  a  part  of  the  team.  The 
MT  should  be  the  primary  vehicle  for  getting  O&S 
requirements  integrated  into  the  system  design. 

g.  Be  cleared  to  all  programs  at  all  levels. 

h.  Be  allowed  to  attend  SPO  staff  meetings. 

i.  Be  required  to  attend  all  meetings,  briefings, 
working  groups,  and  Technical  Interchange  Meetings  (TIMs). 

j.  Be  allowed  to  participate  in  MAA/COEAs,  Multiple 
Aircraft  Reviews  (MARs),  Science  Advisory  Board  (SAB)  as 
required. 

k.  Be  given  the  latitude  (including  TDY  funding)  by 
management  to  consult  with  experts  within  DoD  and  the 
defense  contractors  for  lessons  learned,  or  technical 
assistance . 

l.  Be  allowed  to  use  any  team-building  technique,  such 
as  the  Integrated  Process  Development  Team  (IPDT)  approach 
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to  integrate  themselves  into  the  design  process  and  to 
involve  the  engineers  in  the  maintainability  issues. 

m.  Participate  in  Joint  Integrated  Avionics  Working 
Group  (JIAWG)  and  other  joint  service  activities.  Use 
maintainability  and  ILS  experts  form  other  services  to  the 
maximum  extent  as  possible. 

n.  Establish  and  coordinate  Blue  Two  visits.  Blue  Two 
or  TAG  Reliability  and  Maintainability  visits  can  be  eye¬ 
opening  experiences  for  the  contractors.  These  visits  can 
also  be  an  important  mechanism  to  get  the  engineering 
community  educated  on  the  maintainability  issues. 

o.  Allow  the  User  to  deal  directly  with  the 
contractor,  but  require  that  visit  notifications  be  sent  to 
the  SPO.  Freedom  to  interface  with  the  contractor  can 
afford  them  the  opportunity  to  be  educated  on  the  design, 
understand  what  was  actually  achievable,  and  convey  the  user 
requirements  to  the  contractor. 

p.  Be  allowed  to  obtain  the  acquisition  training  upon 
before  entering  the  SPO.  This  may  require  waivering  grade 
on  courses  such  as  SYS  100,  SYS  200,  SYS  225,  and  SYS  228. 
SPO  management,  the  DPML,  and  the  supporting  command  should 
all  be  responsible  for  making  this  happen. 

7.5  NAVY  INVOLVEMENT. 

7.5.1  THREAT  AND  EFFECTIVENESS  EXPERTS.  The  following 
agencies  and  organizations  can  provide  expertise  in  mission 
scenarios,  threat,  and  effectiveness  analysis: 

7.5.2  O&S  AND  MPTS  EXPERTS.  The  following  agencies  and 
organizations  can  provide  expertise  in  RM&S  and  MPTS: 
NAVAIRSYSCOM/PMA  205-10/PMA  XXX. 

7.5.3  MANAGEMENT . 
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SECTION  8.0 

DISCIPLINED  REQUIREMENTS  GENERATION 

8.0  Disciplined  requirements  generation  means  integrating 
cost,  MPTS,  and  O&S  into  the  system  design  while  it  is  still 
fluid.  This  requires  early  integration  of  the  User  into  the 
acquisition  process,  and  educating  the  User.  It  results  in 
a  match  of  requirements  and  achievable  technologies  to 
baseline  an  executable  program  at  the  start  of  FSED.  It 
also  identifies  PPPI  technologies,  early,  so  that  the  weapon 
system  is  designed  to  defeat  an  evolving  threat.  It  also 
allows  the  User  to  make  informed  decisions. 

The  ATP  SPO  used  a  common  sense  approach  to  system 
design  and  definition  that  required  constant  communication, 
openness  and  trust,  and  interaction  between  the  engineers 
and  the  User  that  had  never  been  achieved  in  the  past.  Some 
of  the  techniques  used  by  the  ATP  SPO  are  as  follows: 
understand  the  TAC  requirements  and  use  the  best  design 
philosophy  to  achieve  them;  educate  and  involve  the  User 
through  the  trade  study  process.  Maintenance  Team  (MT) 
meetings,  and  Training  Planning  Team  (TPT)  meetings;  direct 
the  contractors  to  use  the  same  analytical  tools;  ensure  the 
Government  is  able  to  validate  the  contractors  results; 
ensure  the  Government  defines  the  mission  scenarios  and 
baselines  the  threat  description;  attempt  to  maintain 
commonality  with  existing  O&S  concepts;  and,  ensure  the 
Government  and  Contractor  to  develop  supporting 
documentation.  The  SPO,  therefore,  may  want  to  employ  some 
of  these  concepts  which  are  discussed  in  more  detail  next 
(1;  20;  21;  23;  30;  31;  33;  38). 

8.1  SYSTEM  DEFINITION  AND  DESIGN.  The  SPO  should  attempt 
to  focus  the  D/V  efforts  on  a  single  mission  requirement. 

The  SPO  director  should  push  the  use  of  state-of-the-art 
technology  and  discriminate  application  and  tailoring  of 
military  standards  and  specifications,  if  applicable  (30). 

He  should  also  insist  on  100  percent  commonality  and 
interoperability  with  existing  O&S  and  MPTS  equipment  and 
concepts.  The  trade  study  process  should  require  extensive 
interaction  between  the  technical  people  and  the  User  in 
order  to  match  requirements  with  doable  technologies. 

8.2  TRADE  STUDY  PROCESS .  The  following  process  was 
employed  very  successfully  by  the  ATP  SPO.  and  resulted  in  a 
match  of  requirements  and  achievable  technologies  to 
baseline  an  executable  program  by  FSED.  It  also  identified 
PPPI  concepts  that  were  accounted  for  in  the  design.  The 
process  was  as  follows  (1;  34;  40): 

a.  It  started  out  with  a  number  of  generic  concepts, 
based  on  the  assumption  that  certain  technologies  would  be 
available  at  the  appropriate  time,  and  broadly  stated  User 
requirements. 

b.  Then,  the  User  choose  a  specific  design  concept. 

c.  And  this  is  where  the  education  process  began: 
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(1)  Through  interaction  between  the  engineers  and 
the  User,  the  User  was  educated  on  what  was  doable. 

(2)  Through  analysis  and  testing  the  engineers 
came  up  with  design  concepts,  based  on  the  broadly  stated 
User  requirements,  and  the  associated  cost,  technical, 
manufacturing,  MPTS,  and  O&S  characteristics.  The  cost  and 
technical  data  was  derived  from  a  relative  sensitivity  cost 
model  and  the  TAG  BRAWLER  effectiveness  model . 

(3)  Based  on  this  the  User  made  performance 

tradeoffs . 

(4)  The  above  process  was  repeated  with  the  SPO 
providing  design  alternatives  with  supporting  data  and  the 
user  making  the  performance  tradeoffs. 

d.  The  SPO  also  identified  dates  when  the  avionics, 
aircraft,  and  engine  designs  needed  to  be  finalized. 

e.  The  end  result  was  a  data  base  that  matched 
requirements  with  doable  technologies  (with  growth  designed 
in  for  PPPI  options)  to  baseline  an  executable  program  at 
PSED. 

8.3  INTEGRATED  DESIGN.  By  integrating  the  trade  study 
activities,  with  the  test,  OSS,  and  MPTS  activities  the 
final  design  was  not  only  doable,  but  it  was  an  integrated 
design.  Hence,  the  SPO  should  consider  some  of  the 
following  concepts  when  going  through  this  process. 

8.3.1  TEST  PHILOSOPHY.  Analysis  and  test  play  a  crucial 
role  in  verifying  and  documenting  the  doable  technologies  as 
well  as  the  potential  PPPI  technologies.  The  following  test 
concepts  should  be  considered  in  the  D/V  phase  of  the 
program: 

8. 3. 1.1  TEAM  CONCEPT.  The  test  manager  should  ensure  that 
the  User,  Contractor,  O&S,  MPTS,  AFOTEC,  flight  test  center, 
and  ALD  communities  are  involved  in  PMR,  working  group,  test 
planning,  and  technical  meetings. 

8. 3. 1.2  TEMP.  The  test  manager  should  ensure  that  the 
User,  Contractor,  OSS,  MPTS,  AFOTEC,  flight  test  ''enter,  and 
ALD  communities  are  involved  in  the  preparation  and  review 
of  the  TEMP. 

8. 3. 1.3  CLASS  III  MOCKUPS .  Since  the  prototypes  are, 
typically,  not  available  until  late  in  the  D/V  program,  the 
contractor  should  be  required  to  build  Class  III  mockups  for 
supportability  and  maintai^nability  demonstrations  as  well  as 
the  MPTS  efforts  (38) . 

8. 3. 1.4  ”FLY - BEFORE - BUY ” / PROTOTYPES .  The  SPO  should 
employ  the  "fly-bef ore-buy”  approach  by  testing  with 
prototypes  before  committing  funds  to  FSED. 

8. 3. 1.5  AVIONICS  FLYING  LAB.  The  contractors  should  use  an 
Avionics  Flying  Lab  (AFL)  to  test  avionics  design  concepts 
not  planned  to  be  tested  on  the  prototypes,  if  applicable. 
Otherwise,  the  contractor  should  be  required  to  assemble  a 
full-scale  ground  test. 
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8. 3. 1.6  GROUND  TESTING.  The  contractor  should  be  required 
to  use  ground  testing  to  test  systems,  subsystems,  and 
components  that  will  not  be  tested  in  the  prototype.  Ground 
testing  should  also  be  used  to  support  flight  testing, 
especially  for  the  high-risk  tests. 

8. 3. 1.7  TEST  INFORMATION  SHEETS.  The  SPO  should  require 
draft  Test  Information  Sheets  (TISs)  on  all  tests  to  be 
delivered  at  least  12  months  in  advance  of  flight  testing. 

8. 3. 1.8  ENGINEERING  INSPECTION  TEAMS  (EITsl.  The  SPO 
should  encourage  and  support  independent  Engineering 
Inspection  Teams  (EITs)  for  all  test  activities.  The 
primary  concern  of  these  teams  should  be  to  ensure  that  the 
testing  support  a  higher-level  requirement,  and  that  it 
constitutes  adequate  risk  reduction.  SPO  and  User 
representatives  should  participate  in  these  reviews  as  well. 

8.3.1. 9  GOVERNMENT  FURNISHED  EQUIPMENT  (GFEl.  If  the  SPO 
acquires  GPE  from  other  organizations  it  should  require  a 
written  letter  from  that  SPO  director,  and  at  least  a  90  day 
notice  if  a  problem  with  the  delivery  should  arise.  The  GFE 
list  should  be  reviewed  at  least  once  a  month  by  the  SPO  GFE 
manager . 

8.3.2  Operations  and  Support  (Q&S).  It  is  necessary  for 
the  SPO  to  develop  techniques  which  will  allow  O&S 
requirements  to  be  integrated  into  the  design.  The  SPO  may 
want  to  consider  the  following  concepts  which  may  prevent 
OSS  personnel  from  using  the  results  of  the  design  process 
to  find  OSS  answers  to  a  fixed  design  (30;  33;  38;  41): 

8. 3. 2.1  TEAM  APPROACH.  Assemble  a  team  OF  OSS  experts 
within  the  SPO  to  work  OSS  requirements.  This  team  should 
include  the  SPO,  ATC,  AFLC,  and  User  representatives. 
Representatives  from  TAG  should  be  collocated  in  the  SPO 
too.  The  team  should  develop  a  plan  which  documents  their 
approach  to  disciplined  requirements  generation,  and  they 
should  establish  a  Maintenance  Team  (MT). 

8. 3. 2. 2  OSS  PLAN.  Should  document  their  approach  to 
disciplined  requirements  generation.  Ensures  that  the  OSS 
concerns  and  concepts  are  integrated  into  the  design. 

8. 3. 2. 3  MAINTENANCE  TEAM. 

8. 3. 2. 3.1  MT  PARTICIPANTS.  The  MT  membership  should 
consist  of  designated  representatives  from  each 
participating  agency.  The  following  organizations  should  be 
represented: 

a .  AFSC  members : 

(1)  SPO  members: 

(2)  Flight  test  Center  members: 

b .  TAG  members : 

HQ  TAC/DRB 
/DRF 
/DOT 
/LGQ 
/DRI 

4444  OPS  SQ 
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ASD/TACLO-A 

c .  ATC  Members : 

HQ  ATC/TTO 
3306  TDS 
ASD/TTA 
3752  PLOTS 
3700  TCHTW 

d.  AFLC  Members: 

SM-ALC/LAT 
00-ALC/LIR 
OC-ALC/LPA 
SA-ALC/LPF 
HQ  AFLC/MMC 

e .  AFOTEC  Members : 

HQ/AFOTEC/TEF 

/TEZ 

/LGM 

/OAH 

DET  5,  AFOTEC 

£.  Other  participating  organizations: 
SAF/AQPF/AQPT 

HQ  USAF/XOOT/LEYM/DPPT/PRQE 
APMPC/DPMYP 

NAVAIRSYSCOM/PMA  205-10/PMA  XXX 
AFSCS/SRPT 

g.  Weapon  System  Contractor (s) 

h.  Engine  Contractor(s) 

8. 3. 2. 3. 2  MT  CHARTER.  The  MT  will: 

a.  Assist  and  advise  the  program  director  in 
decisions  made  concerning  the  O&S. 

b.  Coordinate  required  Air  Force  inputs  to  the 
contractor  OSS  development  effort. 

c.  Provide  an  open  forum  to  exchange  information 
on  candidate  OSS  concepts  and  technologies  for 
implementation  in  the  weapon  system,  including 
transition  of  government  Research  S  Development  (RSD) 
efforts  and  providing  inputs  for  the  establishment  of 
priorities  for  OSS  RSD. 

d.  Identify  MAJCOM  activities  required  to  ensure 
timely  development  of  the  OSS  concept  for  weapon  system 
deployment  and  coordinate  accomplishment  of  those 
actions. 

e.  Convene  MT  meetings  and  update  the  plan,  as 
required,  but  at  least  annually. 

8. 3. 2. 3. 3  SPO  RESPONSIBILITIES.  The  SPO,  as  the  agent  of 
the  implementing  command,  should  be  responsible  for  the 
overall  management  of  the  program  and  ensuring  that  the 
OSS  concept  is  integrated  into  the  system  acquisition 
program.  The  SPO  will: 

a.  Chair  the  system  MT. 
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b.  Provide  management  for  the  definition, 
development,  integration,  test,  and  acquisition  of  the 
O&S  concept  and  associated  support. 

c.  Establish  and  maintain  significant  milestone 
schedules  for  the  O&S  concept  based  on  the  PMD. 

d.  Include  the  cost  of  O&S  concept,  associated 
support  equipment,  courseware,  data,  contractor- 
conducted  ISO  efforts,  and  interim  contractor  support 
into  program  cost  estimates. 

8. 3. 2. 3. 4  FLIGHT  TEST  CENTER  (FTCl  RESPONSIBILITIES.  The 

FTC  will: 

a.  Provide  support  to  the  SPO  and  participate  in 
the  acquisition  effort  for  the  O&S  concept  to  include 
design  reviews,  course  readiness  reviews,  and  test  and 
evaluation  activities. 

b.  Identify  FTC  personnel  O&S  requirements  to 
the  SPO  for  interim  contractor  support  to  the  CTF  for 
DT&E  and  lOT&E. 

c.  Provide  Subject  Matter  experts  to  support 
ongoing  assessment/evaluation  of  the  contractor 
development  effort. 

d.  In  conjunction  with  the  SPO,  TAC,  ATC,  and 
AFLC,  develop  plans  for  conducting  DT&E  of  the  O&S 
concept . 

8 . 3 . 2 . 3 . 5  TAC  RESPONSIBILITIES.  TAC  will: 

a.  Represent  the  TAF  in  the  development  of  the 
O&S  concept. 

b.  Provide  O&S  requirements  updates  to  the  SORD 
as  required. 

c.  Provide  facilities  required  at  TAF  bases  to 
support  and  maintain  the  system. 

d.  Provide  support  to  the  SPO  and  participate  in 
the  acquisition  effort  for  the  O&S  concept  to  include 
design  reviews,  course  readiness  reviews,  and  T&E 
activities . 

e.  Provide  SMEs  to  support  ongoing  assessment/ 
evaluation  of  the  contractor  O&S  development  effort. 

f.  Solicit  and  consolidate  inputs  from  TAF  major 
commands . 

g.  In  conjunction  with  the  SPO,  AFFTC,  TAC,  ATC, 
and  AFLC,  develop  plans  for  conducting  DT&E  of  the  O&S 
concept. 

8. 3. 2. 3. 6  ATC  RESPONSIBILITIES.  ATC  will: 

a.  Provide  facilities  required  at  ATC  bases  to 
support  O&S  training. 

b.  Provide  support  to  the  SPO  and  participate  in 
the  acquisition  effort  for  the  training  system  to 
include  design  reviews,  course  readiness  reviews,  and 
T&E  activities. 

c.  Provide  instructional  developers  to  support 
ongoing  assessment/evaluation  of  the  contractor  O&S 
development  effort. 
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d.  Identify  ATC  personnel  training  requirements 
to  the  SPO  for  interim  (Type  1)  training. 

e.  In  conjunction  with  TAG,  determine 
responsibilities  for  delivery  of  maintenance  training 
instruction  (TAG,  ATG,  or  contractor);  budget  and  fund 
for  MTS  operations  and  support  after  RAA  lAW  these 
responsibilities . 

f.  In  conjvinction  with  AFLG  and  TAG,  develop 
plans  for  management  and  operations  and  support 
contracts  for  the  MTS  after  RAA. 

g.  In  conjunction  with  the  SPO,  AFFTG,  TAG,  and 
AFLG,  develop  plans  for  conducting  DT&E  of  the  training 
system. 

8. 3. 2. 3. 7  AFLG  RESPONSIBILITIES.  AFLG  will: 

a.  Provide  O&S  requirements  updates  to  the  SORD 
as  required. 

b.  Provide  facilities  required  at  AFLG  bases  to 
support  and  maintain  the  system. 

c.  Provide  support  to  the  SPO  and  participate  in 
the  acquisition  effort  for  the  O&S  concept  to  include 
design  reviews,  course  readiness  reviews,  and  T&E 
activities . 

d.  Provide  SMEs  to  support  ongoing  -assessment/ 
evaluation  of  the  contractor  development  effort. 

e.  In  conjvuiction  with  the  SPO,  AFFTG,  TAG,  and 
ATG,  develop  plans  for  conducting  DT&E  of  the  O&S 
concept . 

8. 3. 2. 3. 8  Ap'OTEG  RESPONSIBILITIES.  AFOTEG  will: 

a.  Provide  support  to  the  SPO  and  participate  in 
the  acquisition  effort  for  the  O&S  concepts  to  include 
design  reviews,  course  readiness  reviews,  and  T&E 
activities . 

b.  In  conjunction  with  the  SPO,  AFFTG,  TAG,  ATG, 
and  AFLG  develop  operational  plans  to  conduct  both  an 
operational  assessment  and  lOT&E  of  the  O&S  concepts. 

8. 3. 2. 3. 9  WSG  RESPONSIBILITIES.  WSGwill: 

a.  Design,  develop,  integrate,  test,  and  deploy 
the  O&S  concept  lAW  the  weapon  system  specification. 

b.  Provide  interim  contractor  support. 

c.  Maintain  an  Associate  Gontractor  Agreement 
(AGA)  and  Interface  Gontrol  Document  (IGD)  with  the 
engine  contractor. 

d.  Ensure  all  EGPs  on  the  weapon  system  or 
support  system  include  a  proposal  for  required  changes 
to  the  O&S  concept  driven  by  the  EGP. 

8.3.2.3.10  ENGINE  GONTRAGTOR  RESPONSIBILITIES.  The  Engine 
Gontractor  will: 

a.  Design,  develop,  integrate,  test,  and  deploy 
the  weapon  system  Engine  O&S  concept  lAW  the  engine 
specification. 

b.  Provide  interim  contractor  support. 
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c.  Work  with  the  WSC  lAW  Associate  Contractor 
Agreement  (ACA)  and  Interface  Control  Document  (ICD). 

d.  Ensure  all  ECPs  of  the  engine  or  engine 
support  system  include  a  proposal  for  required  changes 
to  the  O&S  concept  driven  by  the  ECP. 

e.  Provide  facility  requirements  for  O&S. 

8. 3. 2. 4  BLUE  TWO  VISITS.  The  SPO  should  coordinate  Blue 
Two  visits. 

8. 3. 2. 5  MOCKUPS .  The  SPO  should  coordinate  maintainability 
demonstrations  on  the  prototype  or  Class  III  mockups  (38). 

8.3.3.  Manpower.  Personnel  &  Training  System  (MPTS) .  It  is 
necessary  for  the  SPO  to  develop  techniques  which  will  allow 
MPTS  requirements  to  be  integrated  into  the  design.  The  SPO 
may  want  to  consider  the  following  techniques  which  may 
prevent  MPTS  personnel  from  using  the  results  of  the  design 
process  to  find  MPTS  answers  to  a  fixed  design  (9;  41): 

8. 3. 3.1  TEAM  APPROACH.  Assemble  a  team  OF  MPTS  experts 
within  the  SPO  to  work  MPTS  requirements.  This  team  should 
be  made  up  of  User  as  well  as  personnel  from  the  Deputy  for 
Training  Systems  (ASD/YW).  The  should  be  located  in  the 
SPO.  They  should  establish  a  Training  Planning  Team  (TPT) 
and  A  System  Training  Plan  (STP). 

8. 3. 3. 2  SYSTEM  TRAINING  PLAN.  Should  document  their 
approach  to  disciplined  requirements  generation.  Ensures 
that  the  MPTS  concerns  and  concepts  are  integrated  into  the 
design. 

8. 3.  '3. 3  TRAINING  PLANNING  TEAM.  The  MPTS  team  should 
develop  a  Training  Planning  Teams  (TPT). 

8. 3. 3. 3.1  TPT  MEMBERSHIP.  The  TPT  membership  should 
consist  of  designated  representatives  from  each 
participating  agency.  The  following  organizations  shall  be 
represented: 

a.  AFSC  members: 

(1)  SPO  members: 

(2)  AFFTC  members: 

b.  TAC  members: 

HQ  TAC/DRB 
/DRF 
/DOT 
/LGQ 
/DRI 

4444  OPS  SQ 
ASD/TACLO-A 

c.  ATC  Members: 

HQ  ATC/TTO 
3306  TDS 
ASD/TTA 
3752  FLDTS 
3700  TCHTW 

d.  AFLC  Members: 

SM-ALC/LAT 

00-ALC/LIR 
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OC-ALC/LPA 
SA-ALC/LPF 
HQ  AFLC/MMC 

e.  AFOTEC  Members: 

HQ/AFOTEC/TEF 

/TEZ 

/LGM 

/OAH 

DET  5,  AFOTEC 

f.  Other  participating  organizations: 

SAF/AQPF/AQPT 

HQ  USAF/XOOT/LEYM/DPPT/PRQE 
AFMPC/DPMYF 

ASD/ YWF/ YML/ YWG/ YWP/ ALH/ SDF 
NAVAIRSYSCOM/PMA  205-10/PMA  XXX 
AFSCS/SRPT 

g.  Weapon  System  Contractor 

h.  Engine  Contractor 

8. 3. 3. 3. 2  TPT  CHARTER.  The  TPT  will: 

a.  Assist  and  advise  the  program  director  in 
decisions  made  concerning  the  training  system. 

b.  Coordinate  required  Air  Force  inputs  to  the 
contractor  training  system  development  effort. 

c.  Provide  an  open  forum  to  exchange  information 
on  candidate  training  concepts  and  technologies  for 
implementation  in  the  training  system,  including 
transition  of  government  Research  &  Development  (R&D) 
efforts  and  providing  inputs  for  the  establishment  of 
priorities  for  training  system  R&D. 

d.  Identify  MAJCOM  activities  required  to  ensure 
timely  training  system  support  for  weapon  system 
deployment  and  coordinate  accomplishment  of  those 
actions . 

e.  Convene  TPT  meetings  and  update  the  plan,  as 
required,  but  at  least  annually. 

f.  Work  with  AF  manpower,  personnel,  and  safety 
organizations  to  ensure  implementation  of  the 
Integrated,  Personnel,  and  Consolidated  Training  and 
Safety  (IMPACTS)  Program. 

8. 3. 3. 3. 3  SPO  RESPONSIBILITIES.  The  SPO,  as  the  agent  of 
the  implementing  command,  should  be  responsible  for  the 
overall  management  of  the  system  training  program  and 
ensuring  that  the  training  program  is  integrated  into  the 
system  acquisition  program.  The  SPO  will: 

a.  Chair  the  system  TPT. 

b.  Provide  management  for  the  definition, 
development,  integration,  test,  and  acquisition  of  the 
training  system  and  associated  support. 

c.  Establish  and  maintain  significant  milestone 
schedules  for  the  training  program  based  on  the  PMD. 
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d.  Include  the  cost  of  training  equipment, 
associated  support  equipment,  courseware,  data, 
contractor-conducted  ISD  efforts,  and  interim 
contractor  training  into  program  cost  estimates. 

8 . 3 . 3 . 3 . 4  FLIGHT  TEST  CENTER  ^FTC)  RESPONSIBILITIES.  The 

FTC  will: 

a.  Provide  support  to  the  SPO  and  participate  in 
the  acquisition  effort  for  the  training  system  to 
include  design  reviews,  course  readiness  reviews,  and 
test  and  evaluation  activities. 

b.  Identify  FTC  personnel  tracing  requirements 

to  the  SPO  for  interim  (Type  1)  training  support  to  the 
CTF  for  DT&E  and  lOT&E. 

c.  Provide  Subject  Matter  experts,  including 
pilots  and  maintainers,  to  support  ongoing 
assessment/evaluation  of  the  contractor  training 
development  effort. 

d.  In  conjunction  with  the  SPO,  TAC,  ATC,  and 
AFLC,  develop  plans  for  conducting  DT&E  of  the  training 
system. 

8 . 3 . 3 . 3 . 5  TAC  RESPONSIBILITIES.  TAC  will: 

a.  Represent  the  TAF  in  the  development  of  the 
training  system. 

b.  Provide  training  system  requirements  updates 
to  the  SORO  as  required. 

c.  Provide  facilities  required  at  TAF  bases  to 
support  pilot  and  maintenance  training. 

d.  Provide  support  to  the  SPO  and  participate  in 
the  acquisition  effort  for  the  training  system  to 
include  design  reviews,  course  readiness  reviews,  and 
TSE  activities. 

e.  Provide  SMEs,  including  pilots  and 
maintainers,  and  instructional  developers  to  support 
ongoing  assessment/evaluation  of  the  contractor 
training  development  effort. 

f.  Establish  a  system  Simulator  Certification 
(SIMCERT)  program. 

g.  Identify  TAF  personnel  training  requirements 
to  the  SPO  for  interim  (Type  1)  training. 

h.  Develop  and  update  estimates  of  TAF  Trained 
Personnel  Required  (TPR)  as  required. 

i.  Budget  and  fund  for  operation  and  support  of 
the  pilot  training  system  after  RAA. 

j.  In  conjunction  with  ATC,  determine 
responsibilities  for  delivery  of  maintenance  training 
instruction  (TAC,  ATC,  or  contractor);  budget  and  fund 
for  MTS  operations  and  support  after  RAA  lAW  these 
responsibilities . 

k.  In  conjunction  with  AFLC,  develop  plans  for 
management  and  operations  and  support  contracts  for  the 
PTS  after  RAA. 
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l.  In  conjunction  with  AFLC  and  ATC,  develop 
plans  for  management  and  operations  and  support 
contracts  for  the  MTS  after  RAA. 

m.  Solicit  and  consolidate  inputs  from  TAF  major 
commands . 

n.  In  conjunction  with  the  SPO,  AFFTC,  TAG,  ATC, 
and  AFLC,  develop  plans  for  conducting  DT&E  of  the 
training  system. 

8 . 3 . 3 . 3 . 6  ATC  RESPONSIBILITIES.  ATC  will: 

a.  Provide  facilities  required  at  ATC  bases  to 
support  maintenance  training. 

b.  Provide  support  to  the  SPO  and  participate  in 
the  acquisition  effort  for  the  training  system  to 
include  design  reviews,  course  readiness  reviews,  and 
T&E  activities. 

c.  Provide  instructional  developers  to  support 
ongoing  assessment/evaluation  of  the  contractor 
training  development  effort. 

d.  Identify  ATC  personnel  training  requirements 
to  the  SPO  for  interim  (Type  1)  training. 

e.  In  conjunction  with  TAC,  determine 
responsibilities  for  delivery  of  maintenance  training 
instruction  (TAC,  ATC,  or  contractor);  budget  and  fund 
for  MTS  operations  and  support  after  RAA  lAW  these 
responsibilities . 

f.  In  conjunction  with  AFLC  and  TAC,  develop 
plans  for  management  and  operations  and  support 
contracts  for  the  MTS  ajfter  RAA. 

g.  In  conjunction  with  the  SPO,  AFFTC,  TAC,  and 
AFLC,  develop  plans  for  conducting  DT&E  of  the  training 
system. 

8. 3. 3. 3. 7  AFLC  RESPONSIBILITIES.  AFLC  will: 

a.  Provide  training  system  requirements  updates 
to  the  SORD  as  required. 

b.  Provide  facilities  required  at  AFLC  bases  to 
support  depot  maintenance  training. 

c.  Provide  support  to  the  SPO  and  participate  in 
the  acquisition  effort  for  the  training  system  to 
include  design  reviews,  course  readiness  reviews,  and 
TSE  activities. 

d.  Provide  SMEs ,  including  depot  maintenance  and 
instructional  personnel  to  support  ongoing  assessment/ 
evaluation  of  the  contractor  training  development 
effort . 

e.  Identify  AFLC  personnel  training  requirements 
to  the  SPO  for  interim  (Type  1)  training. 

f.  Budget  and  fund  for  operation  and  support  of 
the  DTS  after  RAA. 

g.  In  conjunction  with  ATC  and  TAC,  develop  plans 
for  management  and  operations  and  support  contracts  for 
the  MTS  and  PTS  after  RAA. 
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h.  In  conjunction  with  the  SPO,  AFFTC,  TAG,  and 
ATC,  develop  plans  for  conducting  DT&E  of  the  training 
system. 

8 . 3 . 3 . 3 . 8  AFOTEC  RESPONSIBILITIES.  AFOTEC  will: 

a.  Provide  support  to  the  SPO  and  participate  in 
the  acquisition  effort  for  the  training  system  to 
include  design  reviews,  course  readiness  reviews,  and 
T&E  activities. 

b.  Identify  AFLC  personnel  training  requirements 
to  the  SPO  for  interim  (Type  1)  training. 

c.  In  conjunction  with  the  SPO,  AFFTC,  TAG,  ATC, 
and  AFLC  develop  operational  plans  to  conduct  both  an 
operational  assessment  and  lOT&E  of  the  training 
system. 

8. 3. 3. 3. 9  WSC  RESPONSIBILITIES.  WSCwill: 

a.  Design,  develop,  integrate,  test,  and  deploy 
the  training  system  lAW  the  training  system 
specification. 

b.  Operate  and  maintain  the  weapon  system 
training  system  as  required  by  the  Air  Force. 

c.  Provide  interim  (Type  1)  training  until 
training  system  RAA. 

d.  Maintain  an  Associate  Contractor  Agreement 
(ACA)  and  Interface  Control  Document  (ICD)  with  the 
engine  contractor  to  ensure  integration  of  the  engine 
contractor  developed  Organizational  and  Intermediate 
(O&I)  level  training  components  into  the  weapon  system 
training  system. 

e.  Ensure  all  ECPs  on  the  weapon  system  or 
support  system  include  a  proposal  for  required  changes 
to  the  training  system  driven  by  the  ECP. 

f.  Provide  facility  requirements  for  the  PTS, 

MTS,  DTS,  and  TSSC. 

8.3.3.3.10  THE  ENGINE  CONTRACTOR  RESPONSIBILITIES.  The 
Engine  Contractor  will: 

a.  Design,  develop,  integrate,  test,  and  deploy 
the  weapon  system  Engine  Maintenance  Training  System 
(EMTS)  lAW  the  engine  specification. 

b.  Operate  and  maintain  the  weapon  system  EMTS  as 
required  by  the  Air  Force. 

c.  Provide  interim  (Type  1)  training  until  EMTS 

RAA. 

d.  Work  with  the  WSC  lAW  Associate  Contractor 
Agreement  (ACA)  and  Interface  Control  Document  (ICD)  to 
ensure  integration  of  the  EMTS  into  the  weapon  system 
training  system. 

e.  Ensure  all  ECPs  of  the  engine  or  engine 
support  system  include  a  proposal  for  required  changes 
to  the  EMTS  driven  by  the  ECP. 

f.  Provide  facility  requirements  for  DTS. 

8.4  SIMULATION .  Simulation  is  an  analytical  tool  which  can 

support  a  number  of  efforts  that  could  arise  during  the  D/V 
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phase,  such  as  MAA/COEAs,  sensitivity  analyses,  SABs,  CAIGs, 
and  special  studies  directed  by  outside  agencies.  More 
importantly,  they  can  be  used  in  D/V  for  validating  the 
contractor  effectiveness  analysis  results,  the  measuring 
tactical  utility  of  a  given  weapons  system,  or  subsystem 
design.  The  simulation  tools,  such  as  TAG  BRAWLER,  should 
incorporate  the  most  current  baseline  threat,  mission 
scenario,  and  Man- In-The-Loop  (MITL)  data.  Successful 
baselining  and  integration  of  these  activities  will, 
therefore,  add  justification  and  traceability  to  the 
requirements  generation  process.  This  area  should  be  a 
topic  of  the  quarterly  program  reviews.  SPOs  may  want  to 
consider  some  of  the  following  concepts  in  these  areas. 

8.4.1  THREAT .  The  program  director  must  ensure  that  the 
Government  and  the  Contractor  are  using  a  baselined  threat 
discription.  He  must  also  ensure  that  the  User  and  the 
appropriate  participating  agencies  are  involved  in 
baselining  the  threat  description.  In  order  to  accomplish 
this,  the  SPO  should  establish  a  Threat  Working  Group  (TWG) 
to  oversee  this  process. 

8. 4. 1.1  TWG  MEMBERSHIP.  In  addition  to  the  agencies  that 
have  already  been  mentioned,  the  following  agencies  should 
participate  in  the  TWG:  AP/IN,  AFCS/S&A,  Foreign  Technology 
Division  (FTD),  ASD/IN,  ASD/ENSSS,  and  DIA. 

8. 4. 1.2  TWG  CHARTER.  The  TWG  will  (34): 

a.  Ensure  the  threat  baseline  is  established. 

b.  Ensure  the  contractor  is  conforming  to  the  threat 
description. 

c.  Periodically  update  the  threat  baseline. 

d.  Evaluate  the  most  current  data  for  inclusion  into 
the  threat  baseline.  The  sources  of  data  are: 

8 . 4 . 1 . 3  SPO  RESPONSIBILITIES. 

8. 4. 1.4  FTC  RESPONSIBILITIES. 

8. 4. 1.5  TAG  RESPONSIBILITIES. 

8 . 4 . 1 . 6  FTD  RESPONSIBILITIES. 

8. 4.1.7  DIA  RESPONSIBILITIES. 

8. 4.1.8  INTELLIGENCE  COMMUNITY  RESPONSIBILITIES. 

8. 4. 1.9  AFOTEC  RESPONSIBILITIES. 

8.4.1.10  WSC  RESPONSIBILITIES. 

8.4.1.11  THE  ENGINE  CONTRACTOR  RESPONSIBILITIES. 

8.4.2  MISSION  SCENARIOS.  The  program  director  must  ensure 
that  the  Government  and  the  Contractor  are  using  approved 
mission  scenarios  in  the  D/V  activities.  He/she  must  ensure 
that  the  User  and  the  appropriate  participating  agencies  are 
involved  in  the  definition  and  approval  of  the  mission 
scenarios.  In  order  to  accomplish  this,  the  SPO  should 
establish  a  Mission  Scenario  Group  (MSG)  to  oversee  this 
process  (42). 

8. 4. 2.1  MSG  CHARTER.  The  MSG  will: 

a.  Support  the  TWG,  Cockpit  Working  Group,  TAG 
BRAWLER,  and  MITL  as  required. 
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b.  Ensure  the  contractor  is  conforming  to  the  approved 
mission  scenarios  in  his  effectiveness  analysis  and  MITL 
activities . 

c.  Periodically  review  and  update  as  required. 

8. 4. 2. 2  MSG  PARTICIPATION.  The  TWG  membership  should 
consist  of  designated  representatives  from  each 
participating  agency.  The  following  organizations  shall  be 
represented: 


a. 

AFSC  members : 

(1)  SPO  members: 

( 2 )  PTC  members : 

b. 

TAC  members : 

HQ  TAC/DRB 

/DRF 

/DOT 

/LGQ 

/DRI 

4444  OPS  SQ 

ASD/TACLO-A 

c . 

Other  participating  organizations: 

SAF/AQPP/AQPT 

HQ  USAF/XOOT/LEYM/DPPT/PRQE 

AFMPC/DPMYP 

ASD/IN/ENSSS 

NAVAIRSYSCOM/PMA  205-10/PMA  XXX 
AFCS/S&A 

Defense  Intelligence  Agency 

FTD 

d. 

Weapon  System  Contractor 

e . 

Engine  Contractor 

8. 4. 2. 3 

SPO  RESPONSIBILITIES. 

8. 4. 2. 4 

FTC  RESPONSIBILITIES. 

8. 4. 2. 5 

TAC  RESPONSIBILITIES. 

8. 4. 2. 6 

FTD  RESPONSIBILITIES. 

8. 4. 2. 7 

DIA  RESPONSIBILITIES. 

8. 4. 2. 8 

INTELLIGENCE  COMMUNITY  RESPONSIBILITIES. 

8. 4. 2. 9 

AFSC/S&A  RESPONSIBILITIES. 

8.4.2.10 

AFOTEC  RESPONSIBILITIES. 

8.4.2.11 

WSC  RESPONSIBILITIES. 

8.4.2.12 

THE  ENGINE  CONTRACTOR  RESPONSIBILITIES. 

8.4.3  MAN-IN-THE-LOOP  (MITL).  The  SPO  should  establish  an 
MILT  program,  if  applicable. 

8. 4. 3.1  MITL  CHARTER.  The  MILT  should  be  a  subgroup  of  the 
CPWG  (42).  The  group  should  consist  of  at  least  12  test 
pilots  and  12  operational  pilots  who  periodically  fly  the 
simulator  (42).  The  objective  of  the  MITL  activities  should 
be  to  provide  expert  opinions  and  comments  to  the  Government 
and  the  Contractor  on  the  cockpit  development,  the  mission 
scenario  definition,  and  the  tactics  (42). 

8. 4. 3. 2  MITL  PARTICIPATION.  TAP,  TAC,  AFOTEC,  AFSC  FTCs , 
FTD,  and  the  WSC. 
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8. 4. 3. 3  SPO  RESPONSIBILITIES.  The  SPO  will: 

a.  Develop  and  approve  the  list  of  AFSC  and  TAG  pilots 
participating  in  the  MITL  program. 

b.  Develop  and  maintain  the  MITL  schedule. 

c.  Maintain  a  file  of  the  pilot  inputs  associated  with 
the  MITL  program. 

d.  Incorporate  the  inputs  into  the  CPHG  and  MSDG,  or 
the  associated  documentation,  as  required. 

8. 4. 3. 4  FTC  RESPONSIBILITIES.  The  FTCs  will: 

8 . 4 . 3 . 5  TAG  RESPONSIBILITIES.  TAG  will: 

a.  Identify  a  list  of  12  operational  pilots  that  will 
participate  in  the  MITL  program. 

b.  Ensure  the  pilots  are  available  to  support  the 


program. 

8. 4.3. 6  FTP  RESPONSIBILITIES. 

8 . 4 . 3 . 7  AFCS/S&A  RESPONSIBILITIES. 

8 . 4 . 3 . 8  AFOTEC  RESPONSIBILITIES. 

8. 4. 3. 9  WSC  RESPONSIBILITIES.  WSCwill: 


a.  Incorporate  the  inputs  into  the  CPWG  and  MSDG,  or 
the  associated  documentation,  as  required. 

8.4.3.10  ENGINE  CONTRACTOR  RESPONSIBILITIES. 

8.4.4  TAG  BRAWLER.  The  program  director  must  ensure  that 
the  Government  and  the  Contractor  are  using  the  same  version 
of  TAG  BRAWLER.  He/she  must  ensure  that  the  user  and  the 
participating  organizations  are  involved  in  the  definition 
and  the  baselining  of  TAG  BRAWLER  model  for  the  D/V  effort. 
In  order  to  accomplish  this,  the  SPO  should  establish  a  TAG 
BRAWLER  Group  (TBG)  to  oversee  this  process.. 

8. 4. 4.1  TBG  CHARTER.  The  TBG  will: 

a.  Support  the  TWG,  Cockpit  Working  Group,  special 
studies,  and  MITL  as  required. 

b.  Ensure  the  contractor  is  conforming  to  the  approved 
version  in  his  effectiveness  analysis  and  MITL  activities. 

c.  Periodically  review  and  update  TAG  BRAWLER  as 


required. 


8. 4. 4. 2  TBG  PARTICIPATION.  The  TBDG  membership  should 
consist  of  designated  representatives  from:  SPO,  TAG, 
AFCS/SSA,  FTD,  AF/IN,  ASD/ENSSS,  FTCS,  AND  WSCS. 

8 . 4 . 4 . 3  SPO  RESPONSIBILITIES. 


8. 4. 4. 4 

PTC  RESPONSIBILITIES. 

8. 4. 4. 5 

TAG  RESPONSIBILITIES. 

8. 4. 4. 6 

FTD  RESPONSIBILITIES. 

8. 4. 4. 7 

IN  COMMUNITY  RESPONSIBILITIES. 

8. 4.4. 9 

APSC/SSA  RESPONSIBILITIES. 

8.4.4.10 

AFOTEC  RESPONSIBILITIES. 

8.4.4.11 

WSC  RESPONSIBILITIES. 

8.4.4.12  ASD/ENSSS  RESPONSIBILITIES. 

8.4.5  OTHER  SIMULATIONS.  The  SPO  may  want  to  do  an 
independent  check  of  the  prime  contractor's  effectiveness 
study  results  (Mission  Effectiveness  Analysis),  or  do  a 
separate  in-house  study.  Therefore,  it  may  be  necessary  for 
the  SPO  to  approve  the  analytical  tools  that  will  be  used  by 
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other  contractors.  This,  too,  should  be  discussed  in  the 
TBG.  The  same  procedures  for  defining  and  approving  the 
model (s)  should  apply. 

8.5  DOCUMENTATION .  Refer  to  Section  11.0,  titled. 

Synergism  of  Documentation. 

8.6  BUDGET  MODEL.  A  relational,  detailed  budget  model 
should  be  developed  to  validate  the  contractor  data  in 
relative  terms,  and  cost  proposed  configurations  as  they 
were  developed  to  match  the  latest  set  of  requirements  (23; 
35).  The  model  should  not  be  limited  to  just  Air  Force 
data.  Contractor  data  should  be  used  as  well. 

8.7  MOAs  and  Working  Group  Charters .  There  will  be  a  large 
number  of  organizations  supporting  the  program  office.  The 
program  director  should  ensure  that  these  documents  do  not 
restrict  or  limit  the  technical  support  from  these 
participating  agencies.  If  a  MOAs  are  used,  it  should 
address,  but  is  certainly  not  limited  to  the  nximber  of 
people,  their  type  of  expertise....  The  program  director's 
objective  should  be  to  maximize  the  support  from  these 
organizations . 

8.8  TECHNOLOGY  TRANSITION.  In  order  to  minimize  cost  and 
properly  integrate  O&S  and  MPTS  requirements  into  the  design 
the  SPO  has  to  identify  potential  PPPI  improvements  as  early 
as  possible.  These  potential  improvements  have  to  be 
properly  documented  and  tracked.  The  SPO  should  do  this  on 
fact  sheets  (refer  to  Appendix  F).  The  SPO  should  establish 
a  Technology  Panel  and  a  Technology  Transition  Working 
Group.  Also,  the  participating  members  and  their  roles  and 
responsibilities  should  be  established  as  early  as  possible. 
8.8.1  TECHNOLOGY  PANELS.  The  TP  will: 

a.  Review  all  Air  Force  technology  programs  and 
determine  their  applicability  to  the  program. 

b.  Docviment,  track,  and  periodically  review  the  status 
of  the  programs . 


8.8.2  TP  PARTICIPATION  AND  RESPONSIBILITIES. 

8. 8. 2.1  SPO  RESPONSIBILITIES.  The  SPO  will: 

a.  Chair  the  TP. 

b.  Coordinate  and  host  the  meetings,  semiannually. 

c.  Document  and  track  the  action  items  through  fact 
sheets  (Refer  to  Appendix  F). 

d.  Establish  MOAs  as  required. 

8. 8. 2. 2  AFSC  RESPONSIBILITIES.  AFSC  will 

a.  Participate  in  the  TPs. 

b.  Coordinate  Form  56/PMD  documentation,  as  required. 

8. 8. 2. 3  SAF/AOL/AOP  RESPONSIBILITIES.  SAF/AQR/AQL  will: 

a.  Participate  in  TPs. 

b.  Execute  proper  PMD  direction,  as  required. 

8. 8. 2. 4  TAC  RESPONSIBILITIES.  TAC  will: 


a.  Participate  in  TPs. 

b.  Present  technology  roadmap  briefings,  as  required. 

c.  Represent  the  TAF. 

d.  Provide  updates  to  the  SORD  as  required. 
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e.  Sol‘'cit  and  consolidate  inputs  from  TAP  major 
commands . 

f.  In  conjunction  with  the  SPO,  AFFTC,  TAG,  ATC,  and 
AFLC,  develop  plans  for  conducting  DTSE  of  the  O&S  concept. 
8. 8. 2. 5  OTHER  AGENCY  PARTICIPATION.  Other  participating 
organizations  will: 

a.  Participate  in  TPs  as  required. 

b.  Present  status  of  applicable  programs  as  directed 
by  the  SPO. 

c.  Acquire  necessary  program  administrative  changes  as 
required  to  support  the  SPO. 

8.9  CONTRACTOR  INTERFACE  The  contractor  is  responsible  for 
performing  the  trade  studies.  This  process  includes 
acquiring  sufficient  data  to  make  requirements  decisions  and 
maturing  technologies  with  which  to  enter  FSD,  both  risk 
reduction  activities.  This  is  an  iterative  process  in  which 
the  SPO  provides  the  contractors  a  system  requirement 
against  which  the  contractors  are  applying  trade  studies  and 
demonstration  tools,  and  returning  recommendations  for 
changes  to  the  requirement  for  better  system  optimization. 
These  requirements  recommendations  were  analyzed  by  the  Air 
Force.  Accepted  recommendations  are  documented  in  the 
system  specification  and  returned  to  the  contractor  as  new 
requirements.  This  cycle  was  repeated  annually  during  the 
D/V  phase.  The  contractor  will: 

a.  Establish  and  maintain  ACAs  and/or  ICDs  with  the 
appropriate  contractors.  These  will  be  updated  in  FSED. 

b.  Participate  in  PMRs,  Working  groups,  technical 
meetings . . . 


125 


SECTION  9.Q 
TEAM  APPROACH 

<?.0  In  the  case  of  the  ATF  program,  a  successful  team- 
acquisition  strategy  appeared  do  be  a  byproduct  of  good 
communication  and  exchange  of  information  and  a  disciplined 
requirements  generation  approach.  The  ATF  SPO  used 
innovative  management  and  organizational  concepts  in  these 
areas  as  well  as  an  effective  team-acquisition  strategy 
through  working  groups,  definition  groups,  technical 
meetings,  PMRs ,  maintenance  teams,  and  training  team.  They 
accomplished  this  without  a  need  for  MOAs  or  MOUs . 

9.1  WORKING  GROUPS.  Working  Groups  should  be  established 
for  armament,  avionics,  cockpit,  test  planning,  and  threat. 
These  groups  should  consist  of  representatives  from  the 
applicable  government  and  contractor  organizations,  they 
should  be  chaired  by  SPO  management,  and  meet  as  required. 
Their  mission  should  be  to  assist  in  the  development  of  an 
integrated  weapon  system  design. 

9.1.1  ARMAMENT  WORKING  GROUP.  See  Attachment  A.  At  a 
minimum,  the  purpose(s)  of  the  ARWG  should  be  (3): 

a.  Assist  and  advise  the  program  director  in  decisions 
concerning  the  armament  system. 

b.  Aid  the  WSC  in  executing  a  successful  armament 
system  development  program. 

c.  Provide  an  open  forum  to  exchange  information  on 
candidate  technologies  for  the  armament  system  development. 

d.  Allow  the  User  and  experts  from  participating 
agencies  to  participate  in  the  armament  system  development 
effort . 

e.  Inform  peripheral  government  organizations  of 
progress  by  the  WSCs  in  the  armament  system  integration 
effort. 

9.1.2  AVIONICS  WORKING  GROUP.  See  Attachment  B.  At  a 
minimum,  the  purpose(s)  of  the  AWG  should  be  (4): 

a.  Assist  and  advise  the  program  director  in  decisions 
concerning  the  avionics  system. 

b.  Aid  the  WSC  in  executing  a  successful  avionics 
development  program. 

c.  Provide  an  open  forxim  to  exchange  information  on 
candidate  technologies  for  the  avionics  system  development. 

d.  Allow  the  User  and  experts  from  participating 
agencies  to  participate  in  the  avionics  system  development 
effort . 

e.  Inform  peripheral  government  organizations  of 
progress  by  the  WSCs  in  the  avionics  system  integration 
effort . 

9.1.3  COCKPIT  WORKING  GROUP.  If  applicable.  See 
Attachment  C.  At  a  minimum,  the  purpose(s)  of  the  CWG 
should  be  (5) : 

a.  Assist  and  advise  the  program  director  in  decisions 
concerning  the  cockpit. 


126 


y 


b.  Aid  the  WSC  in  executing  a  successful  cockpit 
development  program. 

c.  Provide  an  open  forum  to  exchange  information  on 
candidate  technologies  for  the  cockpit  development. 

d.  Allow  the  User  and  experts  from  participating 
agencies  to  participate  in  the  cockpit  development  effort. 

e.  Inform  peripheral  government  organizations  of 
progress  by  the  WSCs  in  the  cockpit  development  effort. 

9.1.4  THREAT  WORKING  GROUP.  See  Atachment  D.  At  a 
minimum,  the  purpose(s)  of  the  THG  should  be  (12): 

a.  Assist  and  advise  the  program  director  in  decisions 
concerning  the  threat. 

b.  Provide  an  open  forum  to  exchange  information  on 
current  threat  as  well  as  new  threat  information. 

c.  Allow  the  Ucer  and  experts  from  participating 
agencies  to  participate  in  baselining  the  threat. 

d.  Inform  peripheral  government  organizations  of  the 
status  of  the  threat  baselining  effort. 

9.1.5  TEST  PLANNING  WORKING  GROUP.  See  Attachment  E.  At  a 
minimum,  the  purpose(s)  of  the  TPWG  should  be  (11): 

a.  Assist  and  advise  the  program  director  in  decisions 
concerning  T&E. 

b.  Aid  the  WSC  in  executing  a  successful  T&E  program. 

c.  Provide  an  open  forum  to  exchange  information  on 
test  planning  and  execution. 

d.  Allow  the  User  and  experts  from  participating 
agencies  to  participate  in  test  planning. 

e.  Inform  peripheral  government  organizations  of  , 
progress  by  the  WSCs  in  test  planning  and  execution. 

9.2  TECHNICAL  MEETINGS. 

9.2.1  MAINTAINABILITY .  Refer  to  discussion  on  Maintenance 
Team  in  Section  8.0. 

9.2.2  TRAINING.  Refer  to  discussion  on  Training  Planning 
Team  in  Section  8.0. 

9.2.3  MAN-IN-THE-LOOP .  Refer  to  discussion  on  MITL  in 
Section  8.0 

9.2.4  ' TECHNOLOGY  TRAN S I T I ON .  Refer  to  discussion  of 
technology  transition  in  Section  8.0. 

9.3  PROGRAM  REVIEWS.  The  User,  User  representatives,  ALC 
representatives,  HQ  AFSC,  SAF/AQR/AQL,  AFOTEC,  ATC,  and  AFLC 
should  attend  the  reviews. 

9.4  BRIEFINGS. 

9.4.1  ALTERNATIVES. 

9.4.2  Air  Force  Chief  of  Staff. 

9.4.3  TAC/CC. 

9.4.4  AFSC/CC. 

9.4.5  SCIENCE  ADVISORY  BOARD. 

9.4.6  DAB . 
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SECTION  10.0 

EXCLUSION  FROM  ACQUISITION  PROCESS 

10.0  The  following  areas  have  been  discussed,  extensively, 
in  this  handbook.  They  represent  "conunon  sense"  techniques 
that  were  developed  from  lessons  learned,  and  supported  by 
senior  management.  These  areas,  represent  the  starting 
point  for  a  successful  User-interface  approach  as  well  as 
techniques  for  attempting  to  include  the  User  in  the 
acquisition  process. 

10.1  MANAGEMENT  COMMITMENT. 

10.2  TRADE  STUDY  PROCESS.  The  key  is  to  view  this  as  an 
education  process,  and  to  get  the  engineers  and  the  User  to 
communicate . 

10.3  TRAINING.  This  is,  typically,  a  major  problem  for  the 
User.  This  can  be  overcome  through  constant  communication, 
SPO  libraries,  and  assigning  User  personnel  with  SPO 
experience  to  the  SPO.  However,  grade  restrictions  still 
prevent  enlisted  personnel  from  acquiring  the  necessary 
training . 

10.4  EARLY  USER  INVOLVEMENT.  The  representatives  needs  to 
be  in  place  by  the  start  of  D/V.  Also,  Maintenance  and 
Training  Team  concepts  have  proven  to  be  very  successful  in 
educating  the  User,  involving  him/her  in  the  acquisition 
process,  and  integrating  O&S  and  MPTS  requirements  into  the 
system  design. 

10.5  EXPERIENCED  SPO  PERSONNEL. 

10.5.1  O&S  TEAM  OF  EXPERTS. 

10.5.2  MPTS  TEAM  OF  EXPERTS. 

10.5.3  PILOT  EXPERTS.  Can  be  useful  in  education  the  User, 
and  in  supporting  the  CWG,  MITL,  and  Effectiveness 
activities . 

10.5.4  ANALYSIS.  Bring  in  people  from  AFCS/SSA  and 
engineering  division  to  run  the  TAC  BRAWLER  and  analysis 
activities.  They  interface  well  with  ASD/ENSSS. 

10.5.5  THREAT . 

10.6  TEAM  APPROACH.  Refer  to  discussion  in  previous 
section.  Section  9.0. 
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SECTION  11.0 

SYNERGISM  OF  DOCUMENTATION 


11.0  DOCUMENTATION .  The  following  documents  and  analytical 
tools  should  form  the  database  that  will  be  used  by  the  SPO 
to  answer  inquiries,  justify  the  requirements,  the  trade 
study  results,  and  the  PPPI  technologies  (1).  They  should 
ensure  that  there  is  traceability  throughout  the  program 
documentation  as  the  requirements  are  being  refined. 

These  documents  also  form  the  data  base  which  will  be  used 
to  prove  that  there  is  a  match  between  the  requirements  and 
the  associated  technology  (1).  Therefore,  in  order  to 
accomplish  these  things,  the  SPO  should  develop  a  matrix 
which  cross  references  each  requirement  through  all  these 
documents  (38).  In  addition,  the  SPO  should  establish  a 
team  to  periodically  update  the  matrix,  and  ensure  that  the 
PSS  and  the  SORD  match  one-for-one  (38).  The  team  should 
include  SPO,  User  representative,  engineering,  TAC/DRB, 
AFOTEC,  HQ  AFSC,  SAF/AQL/AQR,  ATC,  and  AFLC  personnel  at  a 
minimum  (38). 

11.1  SPO  LIBRARY.  This  can  be  a  useful  training  tool  for 
SPO  personnel  as  well  as  the  User  representatives.  A  SPO 
library  will  also  maintain  some  continuity  in  the  program. 

11.2  PROGRAM  MANAGEMENT  DIRECTION  fPMD).  The  User,  HQ 
AFSC,  and  SAF/AQ  representatives  need  attend  working  groups, 
PMRs,  and  technical  meetings.  This  will  enable  the  SPO  to 
get  the  proper  direction,  quickly,  if  required.  The 
headquarters  personnel  need  to  communicate  with  the  SPO  on  a 
daily  basis,  and  come  TDY  to  the  SPO  at  least  once  a  month. 

11.3  PRELIMINARY  SYSTEM  SPECIFICATION.  The  SPO  and  the 
User  should  ensure  that  the  Draft  PSS  and  the  SORD  match 
one-for-one  before  it  is  sent  to  the  contractors.  The  SPO 
and  the  User  should  periodically  review  the  PSS  and  SORD  to 
see  that  they  still  match.  The  PSS  should  be  updated  when 
the  User  makes  major  tradeoffs  in  the  requirements  as 
described  in  the  trade  study  process.  In  addition,  the  MSG, 
TBG,  working  groups,  technical  meetings,  TPT,  and  MT  should 
document  proposed  changes,  and  incorporate  them  into  thee 
PSS  and  SORD  when  they  are  updated. 

11.4  SYSTEM  OPERATIONAL  REQUIREMENTS  DOCUMENT.  The  SORD 
should  be  updated  after  each  tradeoff  decision  point  rather 
than  at  the  end  of  D/V,  or  a  milestone  decision  point  (23). 

11.5  TRADE  STUDY  RESULTS .  The  contractor  should  reference 
the  PSS  and  SORD  paragraphs  for  each  requirement,  or  trade 
study  initiative.  The  contractor  should  develop  a  control 
chart  for  each  SORD  and  PSS  requirement.  These  charts 
should  also  delineate  the  associated  risk  reduction 
activities  (analysis,  test,..)  required  to  reduce  the  level 
of  risk  so  that  they  can  be  incorporated  into  the  weapon 
system  baseline.  The  contractor  should  also  discuss  any 
requirements  that  are  not  achievable  with  current  or  near- 
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term  technologies  in  the  PMRs.  The  contractor  should  also 
identify  potential  PPPI  technologies. 

11.6  SPECIAL  STUDIES.  Special  studies,  such  as  SABs , 
MAA/COEA,  AFSC,  or  User-directed  studies,  are  likely  to 
occur  through  the  course  of  the  D/V  phase.  The  above 
docvimentation  should  reflect  the  results  from  these  studies, 
if  required.  Also,  most  of  the  data  should  be  available  to 
support  these  studies. 

11.6.1  SCIENCE  ADVISORY  BOARD.  The  SPO  should  not  need  to 
generate  any  additional  data,  or  develop  analytical  tools 
for  this  effort.  The  SPO  should  integrate  the  User,  AFOTEC, 
FTD,  AFCS/SSA,  DIA,  and  AF/IN  into  this  effort  to  coordinate 
the  information. 

11.6.2  MAA/COEA.  The  SPO  should  not  need  to  generate  any 
additional  data  for  this  effort.  The  SPO  should  integrate 
the  User,  AFOTEC,  FTD,  AFCS/S&A,  DIA,  and  AF/IN  into  this 
effort  to  coordinate  the  information.  In  fact,  the  WSC  may 
need  to  be  involved. 

11.6.3  INQUIRIES  FROM  EXTERNAL  SOURCES.  The  SPO  should  be 
able  to  answer  these  inquiries  very  quickly  with  the  matrix 
and  the  database 

11.7  SPECIAL  BRIEFINGS. 

11.7.1  AIR  FORCE  CHIEF  OF  STAFF. 

11.7.2  TAC/CC. 

11.7.3  TAF. 

11.7.4  AFSC/CC. 

11.7.5  SABs. 
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Attachment  A 

ATF  Armament  Working  Group  Charter 
The  charter  was  not  included  in  the  draft  handbook. 
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Attachment  B 

ATF  Avionics  Working  Group  Charter 
The  charter  was  not  included  in  the  draft  handbook. 
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Attachment  C 

ATF  Cockpit  Working  Group  Charter 
The  charter  was  not  included  in  the  draft  handbook. 
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Attachment  D 

ATF  Threat  Working  Group  Charter 
The  charter  was  not  included  in  the  draft  handbook. 
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Attachment  E 

ATF  Test  Planning  Working  Group  Charter 
The  charter  was  not  included  in  the  draft  handbook. 
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